-- Logs begin at Fri 2022-07-15 16:52:02 UTC, end at Fri 2022-07-15 18:09:48 UTC. -- Jul 15 17:26:52.753630 dv-cinder-836973-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 15 17:26:52.762140 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 15 17:26:52 2022] *** Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: nodename: dv-cinder-836973-11-iscsi Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: machine: x86_64 Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: clock source: unix Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: pcre jit disabled Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: detected number of CPU cores: 4 Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: current working directory: / Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: detected binary path: /usr/bin/uwsgi-core Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: your processes number limit is 47680 Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: your memory page size is 4096 bytes Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: detected max file descriptor number: 2048 Jul 15 17:26:52.764447 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: lock engine: pthread robust mutexes Jul 15 17:26:52.765210 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: thunder lock: enabled Jul 15 17:26:52.765210 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 15 17:26:52.765210 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 15 17:26:52.807160 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: Python main interpreter initialized at 0x55c2f15f69c0 Jul 15 17:26:52.807160 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: python threads support enabled Jul 15 17:26:52.807160 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: your server socket listen backlog is limited to 100 connections Jul 15 17:26:52.807160 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: your mercy for graceful operations on workers is 90 seconds Jul 15 17:26:52.807563 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: mapped 403077 bytes (393 KB) for 2 cores Jul 15 17:26:52.807638 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: *** Operational MODE: preforking *** Jul 15 17:26:52.807883 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: *** uWSGI is running in multiple interpreter mode *** Jul 15 17:26:52.807883 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: spawned uWSGI master process (pid: 80457) Jul 15 17:26:52.808097 dv-cinder-836973-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 15 17:26:52.812634 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: spawned uWSGI worker 1 (pid: 80471, cores: 1) Jul 15 17:26:52.812634 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: spawned uWSGI worker 2 (pid: 80472, cores: 1) Jul 15 17:26:52.812634 dv-cinder-836973-11-iscsi devstack@keystone.service[80457]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 15 17:26:58.370955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 17:26:58.370955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: from cryptography.utils import int_from_bytes Jul 15 17:26:58.371726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 17:26:58.371726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: from cryptography.utils import int_from_bytes Jul 15 17:26:58.372651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 17:26:58.372651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: from cryptography.utils import int_from_bytes Jul 15 17:26:58.373596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 17:26:58.373596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: from cryptography.utils import int_from_bytes Jul 15 17:26:58.426182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 17:26:58.426363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 17:26:58.426444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 17:26:58.426561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] command line args: [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 17:26:58.426653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 17:26:58.426822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] command line args: [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 17:26:58.426906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 17:26:58.427008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ================================================================================ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 17:26:58.427099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 17:26:58.427268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ================================================================================ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 17:26:58.427398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] admin_token = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.427513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config_dir = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.427600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] admin_token = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.427707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.427799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_dir = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.427898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] config_source = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.427976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] debug = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] config_source = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] debug = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.428929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_config_append = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_config_append = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.429980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_dir = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_dir = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.430868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431871 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.431957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.432144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.432250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.432364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.432454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.432616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439597 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.439967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] publish_errors = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] publish_errors = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] standard_threads = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] standard_threads = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.440996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.441629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.441701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.441759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.441815 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.441883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_journal = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.441944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_journal = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_json = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_json = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_stderr = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_stderr = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] use_syslog = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] use_syslog = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 17:26:58.442559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.442616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.442676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.442738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.442793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.442858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.442915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.442975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.443993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.444943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.445965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.446988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447500 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.447958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.448129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.448205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.448388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.448491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.448676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.448807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.448921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.449821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.450926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.451996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.452089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.452245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.452328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.452520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.452626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.452780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.452912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.453852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.454967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.455919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.456071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.456154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.456358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.456458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.456672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.456753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.456930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.457937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.458988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.459139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.459375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.459451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.459607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.459683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.459838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.459962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460300 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.460979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.461986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.462161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.462257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.462421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.462510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.462664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.462751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.external = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.462916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.password = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.463975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] auth.token = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.464093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.464236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.464333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.external = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.464479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.464567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.464775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.464864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.password = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] auth.token = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.465982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.466969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.467934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.468990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.469999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470886 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.470964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.471937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.472091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.472174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.472878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.472958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.473952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.474916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.475875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.476850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.477982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.478954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.479972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.480148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.480227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.480381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.480459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.480650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.480733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.480901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.485953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.486974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.487843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.488522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.488634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.488726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.488804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.488875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.488947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.489962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490400 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490827 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.490977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.491934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492827 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.492970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.493933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.494889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.495869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496349 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.496859 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.497977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.498980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.499059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.499229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.499308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.499477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.499558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.499751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.499831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.500839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.501986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.502161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.502240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.502418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.502515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.502669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.502892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.502976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.503997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.504975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.505936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.506919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.507899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.508904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509300 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.509883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.510838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.511977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.512950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.513992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.514937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.515094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.515173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.515253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 17:26:58.515329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.515568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.515804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.516057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.516297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 17:26:58.516471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 17:26:58.546617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.546777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.546868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.546943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.547068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.547154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.556080 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.556174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.584185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.584413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.584555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.584793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.637512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.725106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.761067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.761244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.778121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.778355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.828147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.828381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.828518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.828634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.828778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.828862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.829005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.829102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.829226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.829309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.829445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.829662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.854211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.854449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.854571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.854682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.854808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.854909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.855927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.866739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.867381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.879346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.879585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.879808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.879996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.880916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.881138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.881357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 17:26:58.898698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.898932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 17:26:58.899033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.899251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 17:26:58.899333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.899528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 17:26:58.899809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.899900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.900020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 17:26:58.900102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.900300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.900383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.900571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.900694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.900860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.901120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.913028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.913308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.913580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.913862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.914435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.914711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.914981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.915260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.922172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.923541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.925568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.925855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.926141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.926426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.926711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.926877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.926960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.927159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.927250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.927442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.927525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.927722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.927857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.928003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.928094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.928268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.928380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.928724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.929001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.929272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.929544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.929813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.954824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.956207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.956456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.957272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.958362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.958641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.962588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.962870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.963155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.963937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.964214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.964495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.970621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.970906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.971784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.972064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.975971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.976251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.976538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.976849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.977133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.977276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:58.977414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.977562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.977689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.977846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.977960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.978130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.978407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.978681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.978951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.979217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.996742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.997023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.997990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:58.998267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.001575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.002786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.005681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.005982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.006474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.006770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.011193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.011733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.014833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.015348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.018497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.018799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.019598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.019754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.019902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.020051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.020191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.020332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.020481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.020628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.020923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.021206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.040418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.040738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.040888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.041011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.041172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.041282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.041530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.041801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.049259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.049708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.051043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.051323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.051457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.051703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.054738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.054837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.055034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.055128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.055321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.055404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.055600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.055684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.055887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.055972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.069414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.069511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.069695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.069794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.069978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.070062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.070259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.070342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.070539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.070627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.070818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.070904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.071094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.071174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.089574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.090876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.092857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.094189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.096204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.098169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.098443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.099810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.100084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.100366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.100673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.101643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.101929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.102218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.108071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.109656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.109745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.111290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.113447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.113737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.114026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.114300 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.115091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.115383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.115668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.115952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.123823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.124110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.124389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.125238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.125529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.125814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.132795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.133074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.133352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.133621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.133893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.134168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.134313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 17:26:59.134453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.134597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.134723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.134886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.134997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.135165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.135248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.135437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.135528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.135729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.135811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.136014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.136096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.136275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.136359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.136676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.136955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.137244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.137514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.137794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.138063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 17:26:59.175148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c2f15f69c0 pid: 80472 (default app) Jul 15 17:26:59.176647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c2f15f69c0 pid: 80471 (default app) Jul 15 17:26:59.179010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ff36c-4544-47e8-8c7d-faa819943d00 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:26:59.179310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ff36c-4544-47e8-8c7d-faa819943d00 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:26:59.179593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ff36c-4544-47e8-8c7d-faa819943d00 None None] PATH_INFO: `/v3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:26:59.183806 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Fri Jul 15 17:26:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:07.877479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a957-dc20-4680-aec2-657770320531 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:07.878043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a957-dc20-4680-aec2-657770320531 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:07.878043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a957-dc20-4680-aec2-657770320531 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:07.883008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:07.888515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:07.888927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:07.889115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:07.956560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: INFO dbcounter [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] Registered counter for database keystone Jul 15 17:27:07.977635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG oslo_db.sqlalchemy.engines [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80472) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 17:27:07.983851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writer thread running {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 17:27:08.159277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG passlib.handlers.bcrypt [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80472) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 17:27:08.159660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG passlib.handlers.bcrypt [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80472) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 17:27:08.214064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:08.215677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:08.341021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 15 17:27:08.524179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7a957-dc20-4680-aec2-657770320531 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:08.594224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: INFO dbcounter [None req-b1a7a957-dc20-4680-aec2-657770320531 None None] Registered counter for database keystone Jul 15 17:27:08.613831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG oslo_db.sqlalchemy.engines [None req-b1a7a957-dc20-4680-aec2-657770320531 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80471) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 17:27:08.619982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writer thread running {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 17:27:08.737275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7a957-dc20-4680-aec2-657770320531 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:08.742322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94b9c-ec14-4f5a-bba6-de1b3c2cffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:08.743003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94b9c-ec14-4f5a-bba6-de1b3c2cffda admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:08.743297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94b9c-ec14-4f5a-bba6-de1b3c2cffda admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:08.743581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94b9c-ec14-4f5a-bba6-de1b3c2cffda admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:08.752058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f94b9c-ec14-4f5a-bba6-de1b3c2cffda admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:08.804249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1f94b9c-ec14-4f5a-bba6-de1b3c2cffda admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 17:27:08.804623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f94b9c-ec14-4f5a-bba6-de1b3c2cffda admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:08.814420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Fri Jul 15 17:27:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 17:27:08.820430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:08.835255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7317b-6c5f-4b3a-be08-50cfc4c3c6bf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:08.839649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc769155-3420-41d0-9ac2-43085c957e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:08.840804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc769155-3420-41d0-9ac2-43085c957e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:08.841099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc769155-3420-41d0-9ac2-43085c957e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:08.841379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc769155-3420-41d0-9ac2-43085c957e16 admin admin] PATH_INFO: `/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:08.842877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc769155-3420-41d0-9ac2-43085c957e16 admin admin] RBAC: Authorizing `identity:get_project(project_id=6ce7dfbb170044b796f3aa09ded4d2e4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:08.896368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cc769155-3420-41d0-9ac2-43085c957e16 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 17:27:08.896954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc769155-3420-41d0-9ac2-43085c957e16 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:08.901426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Fri Jul 15 17:27:08 2022] GET /identity/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:10.909087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802b48-06ef-403b-87a5-ddaba43c3927 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:10.909730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802b48-06ef-403b-87a5-ddaba43c3927 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:10.909730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802b48-06ef-403b-87a5-ddaba43c3927 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:10.914432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:10.919848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818668-7979-4321-84f8-cc611098258a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:10.920248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818668-7979-4321-84f8-cc611098258a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:10.920421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818668-7979-4321-84f8-cc611098258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:10.943307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-97818668-7979-4321-84f8-cc611098258a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:10.944904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-97818668-7979-4321-84f8-cc611098258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:10.952902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 17:27:10.979180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802b48-06ef-403b-87a5-ddaba43c3927 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:10.991301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802b48-06ef-403b-87a5-ddaba43c3927 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:10.995512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa3793-b59e-4213-a04d-172ef9e62709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:10.996152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa3793-b59e-4213-a04d-172ef9e62709 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:10.996443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa3793-b59e-4213-a04d-172ef9e62709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:10.996762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa3793-b59e-4213-a04d-172ef9e62709 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:11.004719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa3793-b59e-4213-a04d-172ef9e62709 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:11.006518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa3793-b59e-4213-a04d-172ef9e62709 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:11.015331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Fri Jul 15 17:27:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:27:11.021371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818668-7979-4321-84f8-cc611098258a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:11.030736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818668-7979-4321-84f8-cc611098258a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:11.035819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbcd94-9388-48aa-a6cf-9e2cdc7b8894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:11.036717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbcd94-9388-48aa-a6cf-9e2cdc7b8894 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:11.037012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbcd94-9388-48aa-a6cf-9e2cdc7b8894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:11.037299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbcd94-9388-48aa-a6cf-9e2cdc7b8894 admin admin] PATH_INFO: `/v3/users/4a4044f3af624daa8f53a39108ebc20d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:11.038710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbcd94-9388-48aa-a6cf-9e2cdc7b8894 admin admin] RBAC: Authorizing `identity:get_user(user_id=4a4044f3af624daa8f53a39108ebc20d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:11.040555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbcd94-9388-48aa-a6cf-9e2cdc7b8894 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:11.042167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Fri Jul 15 17:27:11 2022] GET /identity/v3/users/4a4044f3af624daa8f53a39108ebc20d => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:13.149928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec428f-871b-4e49-8d16-1edf515f9f2a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.150463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec428f-871b-4e49-8d16-1edf515f9f2a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.150463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec428f-871b-4e49-8d16-1edf515f9f2a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.151832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:13.157359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a3a51-3770-4e33-b631-1884ce09054a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.157776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a3a51-3770-4e33-b631-1884ce09054a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.158064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a3a51-3770-4e33-b631-1884ce09054a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.174306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c40197-fffc-4c85-ae98-9d675cb910cb None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.174655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c40197-fffc-4c85-ae98-9d675cb910cb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.175409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-087a3a51-3770-4e33-b631-1884ce09054a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:13.175518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c40197-fffc-4c85-ae98-9d675cb910cb None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.176614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-087a3a51-3770-4e33-b631-1884ce09054a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:13.176798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:13.181889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.182182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.182466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.222354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG passlib.handlers.bcrypt [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80471) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 17:27:13.222765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG passlib.handlers.bcrypt [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80471) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 17:27:13.261477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:13.269971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:13.270668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:13.288247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a3a51-3770-4e33-b631-1884ce09054a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.313671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:13.376711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:13.399310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.404946 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a3a51-3770-4e33-b631-1884ce09054a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:13.439076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:13.439870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.440158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.440437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.452090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:13.463195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:13.464283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:13.465327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:13.470251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.505720 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb502c5d-34fb-4eb2-82c9-eb2bf9b6286d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:13.508764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:13.547061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:13.547975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.548267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.548550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.555641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:13.574079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093499d-2714-42ef-8f33-14ebd1ea413e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:13.607820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:13.608498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.608816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.609103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.609863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:13.610176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:13.611595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:13.617164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:13.618691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:13.623537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:13.629139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.716294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:27:13.722442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.725162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:13.726287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101e36-ddb0-4604-9471-25f7596cfc55 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:13.760949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:13.761828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.762121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.762402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] PATH_INFO: `/v3/users/4a4044f3af624daa8f53a39108ebc20d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.763635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] RBAC: Authorizing `identity:get_user(user_id=4a4044f3af624daa8f53a39108ebc20d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:13.765171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:13.766812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/users/4a4044f3af624daa8f53a39108ebc20d => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:13.772288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.786866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:13.822267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20e1d7-1f91-4a5d-a2c5-9920b7e499df None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:13.856427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:13.857313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.857604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.857883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.870209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d381765-5ff2-4c10-88e3-309014a34808 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:13.877452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:13.878929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:27:13.899876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:13.900806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:13.901579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:27:13.901579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:27:13.901579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:27:13.901579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:13.901579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:13.904309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:13.905152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:13.905445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:13.905731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:13.906936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:13.907043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.908418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:13.914194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:13.916245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:13.965166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:13.995720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:14.008982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4aba52-fa7c-4fb7-aa2f-b20d8f6406f4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:14.014309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655a1fa-74fa-4d2d-988b-e2bf66b2495a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:14.043807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7c84-c9bd-4dc0-9c9f-960daa723090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:14.044470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7c84-c9bd-4dc0-9c9f-960daa723090 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:14.044789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7c84-c9bd-4dc0-9c9f-960daa723090 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:14.045071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7c84-c9bd-4dc0-9c9f-960daa723090 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:14.045807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7c84-c9bd-4dc0-9c9f-960daa723090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:14.046115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7c84-c9bd-4dc0-9c9f-960daa723090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:14.047510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7c84-c9bd-4dc0-9c9f-960daa723090 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:14.049022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054d45e-0e9c-4dbb-8b57-c48557515c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:14.049680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d45e-0e9c-4dbb-8b57-c48557515c47 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:14.049993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d45e-0e9c-4dbb-8b57-c48557515c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:14.050277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054d45e-0e9c-4dbb-8b57-c48557515c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:14.051109 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d45e-0e9c-4dbb-8b57-c48557515c47 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.domain.id=default, user.id=4a4044f3af624daa8f53a39108ebc20d) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:14.051431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d45e-0e9c-4dbb-8b57-c48557515c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:14.052895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054d45e-0e9c-4dbb-8b57-c48557515c47 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:14.059474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/role_assignments?user.id=4a4044f3af624daa8f53a39108ebc20d&scope.domain.id=default&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:14.065224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:16.400512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4e5a-3ef8-47d7-ad4d-24b780f961d2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:16.401557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4e5a-3ef8-47d7-ad4d-24b780f961d2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:16.402311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4e5a-3ef8-47d7-ad4d-24b780f961d2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:16.403891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:16.409239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3de495-6c9e-4d2b-9150-3aff6b81336d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:16.410080 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3de495-6c9e-4d2b-9150-3aff6b81336d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:16.410559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3de495-6c9e-4d2b-9150-3aff6b81336d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:16.437651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ef3de495-6c9e-4d2b-9150-3aff6b81336d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:16.439207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ef3de495-6c9e-4d2b-9150-3aff6b81336d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:16.527250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 11/23] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:16.549056 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d0820-f03c-4569-8bf6-97bf5edf1e9d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:16.549659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d0820-f03c-4569-8bf6-97bf5edf1e9d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:16.550697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d0820-f03c-4569-8bf6-97bf5edf1e9d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:16.552281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:16.556542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3de495-6c9e-4d2b-9150-3aff6b81336d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:16.579352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d76f9a-2143-4dbe-85cf-347502f17939 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:16.579960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d76f9a-2143-4dbe-85cf-347502f17939 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:16.580440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d76f9a-2143-4dbe-85cf-347502f17939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:16.609093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f4d76f9a-2143-4dbe-85cf-347502f17939 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:16.611671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f4d76f9a-2143-4dbe-85cf-347502f17939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:16.649353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:16.744221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:16.747635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3de495-6c9e-4d2b-9150-3aff6b81336d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:16.783671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:16.786034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d76f9a-2143-4dbe-85cf-347502f17939 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:16.789368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:16.794032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:16.794542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:16.815861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:16.825755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:16.826637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:16.827378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 12/26] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:16.833781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:16.874749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:16.904334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d76f9a-2143-4dbe-85cf-347502f17939 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:16.985399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c98de-f9c0-4c7a-9ecb-85f72900503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:16.986228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c98de-f9c0-4c7a-9ecb-85f72900503f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:16.986525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c98de-f9c0-4c7a-9ecb-85f72900503f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:16.986810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c98de-f9c0-4c7a-9ecb-85f72900503f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:16.989551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ca839-abe3-4fed-a2a5-f5cc133fdb9e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:16.997667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:17.007385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c98de-f9c0-4c7a-9ecb-85f72900503f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:17.009148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c98de-f9c0-4c7a-9ecb-85f72900503f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:17.038914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359cf0d-cdfb-4174-a5e0-a433726b4fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:17.039925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359cf0d-cdfb-4174-a5e0-a433726b4fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.040448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359cf0d-cdfb-4174-a5e0-a433726b4fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.040948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359cf0d-cdfb-4174-a5e0-a433726b4fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.041928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359cf0d-cdfb-4174-a5e0-a433726b4fb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:17.042512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359cf0d-cdfb-4174-a5e0-a433726b4fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:17.044244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359cf0d-cdfb-4174-a5e0-a433726b4fb5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-766c98de-f9c0-4c7a-9ecb-85f72900503f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:27:17.049176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:17.050631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:17.051423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:27:17.051423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:27:17.051423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:27:17.051423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:17.059047 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 15/27] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:17.060860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5086b61-5c3f-4cbe-9c60-30d419ecf2d7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.061383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5086b61-5c3f-4cbe-9c60-30d419ecf2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.061862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5086b61-5c3f-4cbe-9c60-30d419ecf2d7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.063374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 16/28] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:17.065016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5086b61-5c3f-4cbe-9c60-30d419ecf2d7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:17.075186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 13/29] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:17.076782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bead0c-dc7e-4725-96a4-aec007f839cf None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.077079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bead0c-dc7e-4725-96a4-aec007f839cf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.077364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bead0c-dc7e-4725-96a4-aec007f839cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.118967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-96bead0c-dc7e-4725-96a4-aec007f839cf None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:17.124457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-96bead0c-dc7e-4725-96a4-aec007f839cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:17.238967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5086b61-5c3f-4cbe-9c60-30d419ecf2d7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:17.250566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 14/30] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:17.251594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bead0c-dc7e-4725-96a4-aec007f839cf None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:17.288052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:17.301840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a94fe-fca7-47d1-bb31-fc8d6ba63f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:17.302744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a94fe-fca7-47d1-bb31-fc8d6ba63f20 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.303169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a94fe-fca7-47d1-bb31-fc8d6ba63f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.303638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a94fe-fca7-47d1-bb31-fc8d6ba63f20 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.304581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a94fe-fca7-47d1-bb31-fc8d6ba63f20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:17.305011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a94fe-fca7-47d1-bb31-fc8d6ba63f20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:17.306773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a94fe-fca7-47d1-bb31-fc8d6ba63f20 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:17.329811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:17.333993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5243bce-357d-4861-a8f0-1ef525346ea1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.334482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5243bce-357d-4861-a8f0-1ef525346ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.334924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5243bce-357d-4861-a8f0-1ef525346ea1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.337589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 18/32] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:17.342835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5243bce-357d-4861-a8f0-1ef525346ea1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:17.401512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:17.406437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bead0c-dc7e-4725-96a4-aec007f839cf None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:17.459761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:17.460838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.461207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.461852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] PATH_INFO: `/v3/users/4a4044f3af624daa8f53a39108ebc20d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.464396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] RBAC: Authorizing `identity:get_user(user_id=4a4044f3af624daa8f53a39108ebc20d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:17.466260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:17.471718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 15/33] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:17 2022] GET /identity/v3/users/4a4044f3af624daa8f53a39108ebc20d => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:17.471718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:17.488313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5243bce-357d-4861-a8f0-1ef525346ea1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:17.525998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc2e1e-1681-446c-8d64-4ad965a6d1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:17.526686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc2e1e-1681-446c-8d64-4ad965a6d1cd None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.527240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc2e1e-1681-446c-8d64-4ad965a6d1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.527522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc2e1e-1681-446c-8d64-4ad965a6d1cd None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.528353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc2e1e-1681-446c-8d64-4ad965a6d1cd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:17.530043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc2e1e-1681-446c-8d64-4ad965a6d1cd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:17.599174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3606dc-7ff4-4082-b707-07832a3316bf None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:17.634570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:17.635372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.635664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.635946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.636694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:17.638173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:17.747794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 19/34] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 15 17:27:17 2022] POST /identity/v3/roles => generated 225 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:17.749303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e307a-7ba7-409e-816a-a2140da10ca9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.749609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e307a-7ba7-409e-816a-a2140da10ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.749897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e307a-7ba7-409e-816a-a2140da10ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.779119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-351e307a-7ba7-409e-816a-a2140da10ca9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:17.781296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-351e307a-7ba7-409e-816a-a2140da10ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:17.833930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:17.868717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 16/35] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:27:17 2022] POST /identity/v3/projects => generated 301 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:17.868717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:17.874046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 20/36] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:17.875928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4563-ab40-4220-aaf6-064fbfbeb6da None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.877860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4563-ab40-4220-aaf6-064fbfbeb6da None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.877860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae4563-ab40-4220-aaf6-064fbfbeb6da None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.886638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 21/37] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:17.890544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab962f8c-76b1-4803-b9fa-3e6e550f2a62 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:17.890544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab962f8c-76b1-4803-b9fa-3e6e550f2a62 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:17.890544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab962f8c-76b1-4803-b9fa-3e6e550f2a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:17.921881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ab962f8c-76b1-4803-b9fa-3e6e550f2a62 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:17.923070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:17.923197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ab962f8c-76b1-4803-b9fa-3e6e550f2a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:18.008531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 22/38] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:18.009230 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab962f8c-76b1-4803-b9fa-3e6e550f2a62 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:18.027373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff0d6a-6dba-4c25-bccf-6fe069dd5a16 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:18.075874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f1880-f698-4ba7-afa1-ca160cf6f857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:18.076766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f1880-f698-4ba7-afa1-ca160cf6f857 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.077004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f1880-f698-4ba7-afa1-ca160cf6f857 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.077281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f1880-f698-4ba7-afa1-ca160cf6f857 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.078624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f1880-f698-4ba7-afa1-ca160cf6f857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:18.080255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f1880-f698-4ba7-afa1-ca160cf6f857 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:18.082907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 17/39] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:18.083956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053fa3b-7b17-465d-8f77-c276d3f8598d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.084248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053fa3b-7b17-465d-8f77-c276d3f8598d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.084535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053fa3b-7b17-465d-8f77-c276d3f8598d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.085802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 18/40] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:18.086483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053fa3b-7b17-465d-8f77-c276d3f8598d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:18.147934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab962f8c-76b1-4803-b9fa-3e6e550f2a62 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:18.156342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:18.183205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:18.183887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.184182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.184471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.184936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:18.185268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:18.186802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:18.201063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053fa3b-7b17-465d-8f77-c276d3f8598d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:18.235451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc2102-f516-4d90-9e69-8709c76cc4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:18.236151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc2102-f516-4d90-9e69-8709c76cc4c4 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.236442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc2102-f516-4d90-9e69-8709c76cc4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.236753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc2102-f516-4d90-9e69-8709c76cc4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.237587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc2102-f516-4d90-9e69-8709c76cc4c4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:18.238986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc2102-f516-4d90-9e69-8709c76cc4c4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:18.295538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 23/41] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 15 17:27:18 2022] POST /identity/v3/roles => generated 219 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:18.296361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:18.336110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 19/42] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 15 17:27:18 2022] POST /identity/v3/roles => generated 223 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:18.336110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c82f1c-7db7-4180-b7c2-0d9afcbc25da None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.336110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c82f1c-7db7-4180-b7c2-0d9afcbc25da None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.336110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c82f1c-7db7-4180-b7c2-0d9afcbc25da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.356301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e3c82f1c-7db7-4180-b7c2-0d9afcbc25da None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:18.358530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e3c82f1c-7db7-4180-b7c2-0d9afcbc25da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:18.444916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 20/43] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:18.462395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdb216-bea1-4391-932f-47afa8133162 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:18.472508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c82f1c-7db7-4180-b7c2-0d9afcbc25da None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:18.506676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:18.508048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.508344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.509425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] PATH_INFO: `/v3/domains/default/users/4a4044f3af624daa8f53a39108ebc20d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.511578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4a4044f3af624daa8f53a39108ebc20d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:18.513320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:18.562305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:18.586129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c82f1c-7db7-4180-b7c2-0d9afcbc25da None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:18.594475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 24/44] 10.222.0.57 () {58 vars in 1361 bytes} [Fri Jul 15 17:27:18 2022] PUT /identity/v3/domains/default/users/4a4044f3af624daa8f53a39108ebc20d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:18.623804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97881c84-86de-440b-9263-55d2b9389bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:18.624649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97881c84-86de-440b-9263-55d2b9389bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.624946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97881c84-86de-440b-9263-55d2b9389bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97881c84-86de-440b-9263-55d2b9389bd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.626507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97881c84-86de-440b-9263-55d2b9389bd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:18.628580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97881c84-86de-440b-9263-55d2b9389bd1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:18.630336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 21/45] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:18.636113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:18.666425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:18.746124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343512ca-dbb9-4310-970e-3e8b3f6f27fe None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:18.779698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60294ac1-737a-4ec3-b750-c4c5bf75698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:18.780371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60294ac1-737a-4ec3-b750-c4c5bf75698a None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:18.780741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60294ac1-737a-4ec3-b750-c4c5bf75698a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:18.781032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60294ac1-737a-4ec3-b750-c4c5bf75698a None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:18.781753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60294ac1-737a-4ec3-b750-c4c5bf75698a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:18.783224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60294ac1-737a-4ec3-b750-c4c5bf75698a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:18.861902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 25/46] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:27:18 2022] POST /identity/v3/projects => generated 312 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:20.609910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae6b8-5910-41bb-89c8-169fd42d22b5 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:20.610492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae6b8-5910-41bb-89c8-169fd42d22b5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:20.610492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae6b8-5910-41bb-89c8-169fd42d22b5 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:20.611891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 22/47] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:20.617152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f3675-dad3-4d5e-a11b-51b0a99b1792 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:20.617563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f3675-dad3-4d5e-a11b-51b0a99b1792 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:20.617850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f3675-dad3-4d5e-a11b-51b0a99b1792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:20.741683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c64f3675-dad3-4d5e-a11b-51b0a99b1792 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:20.742901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c64f3675-dad3-4d5e-a11b-51b0a99b1792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:20.808218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 26/48] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:20.831654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ae6b8-5910-41bb-89c8-169fd42d22b5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:20.879937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491d4e4-ae24-43ed-95c2-72f00a7ca939 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:20.880161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491d4e4-ae24-43ed-95c2-72f00a7ca939 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:20.880453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491d4e4-ae24-43ed-95c2-72f00a7ca939 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:20.881947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 27/49] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:20.886636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751255f-85c9-4d5a-88d0-8118a69984b6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:20.886937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751255f-85c9-4d5a-88d0-8118a69984b6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:20.887228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751255f-85c9-4d5a-88d0-8118a69984b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:20.904218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f751255f-85c9-4d5a-88d0-8118a69984b6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:20.905455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f751255f-85c9-4d5a-88d0-8118a69984b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:20.907483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:20.933765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ae6b8-5910-41bb-89c8-169fd42d22b5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:20.968820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:20.969611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:20.969897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:20.970170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:20.974178 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 28/50] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:20.980145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:20.987808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:20.988673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:20.989438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 23/51] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:20.994648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751255f-85c9-4d5a-88d0-8118a69984b6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:20.997205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:21.001290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:21.093094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751255f-85c9-4d5a-88d0-8118a69984b6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:21.096979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c36a7-3480-4f32-8ea1-416de726ed73 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:21.127577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:21.128320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:21.128629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:21.128911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:21.129665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:21.130021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:21.130873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:21.131382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:21.131675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:21.131968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:21.132251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:21.133490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:21.134951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:21.136425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:21.136803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 24/52] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:21.139917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 29/53] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:21.142428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:21.145388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:21.213904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:21.249411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f935cd5-4bf1-4852-9885-5e4e3557322c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:21.286618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:21.287515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:21.287807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:21.288089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] PATH_INFO: `/v3/users/4a4044f3af624daa8f53a39108ebc20d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:21.289379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] RBAC: Authorizing `identity:get_user(user_id=4a4044f3af624daa8f53a39108ebc20d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:21.290956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:21.292683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 30/54] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:21 2022] GET /identity/v3/users/4a4044f3af624daa8f53a39108ebc20d => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:21.298611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:21.357457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd4c8-7096-44be-9502-716526f4acac None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:21.370307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:21.407282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa69b65-17c6-433c-a772-a48c52641c4e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:21.412625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a28e3d-66f9-4697-9f9d-1fa84d9121cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:21.413342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a28e3d-66f9-4697-9f9d-1fa84d9121cb None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:21.413635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a28e3d-66f9-4697-9f9d-1fa84d9121cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:21.413917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a28e3d-66f9-4697-9f9d-1fa84d9121cb None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:21.415434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a28e3d-66f9-4697-9f9d-1fa84d9121cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:21.416225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a28e3d-66f9-4697-9f9d-1fa84d9121cb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:21.444560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:21.445432 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:21.445732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:21.446022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:21.447235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:21.448760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:21.450411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 31/55] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:21.456217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:21.530456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 25/56] 10.222.0.57 () {60 vars in 1140 bytes} [Fri Jul 15 17:27:21 2022] POST /identity/v3/projects => generated 298 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:21.556507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a0255-9d06-45fa-b284-dafac597c284 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:21.596032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ad417-df11-44c1-bc97-1011b80ea024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:21.596779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ad417-df11-44c1-bc97-1011b80ea024 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:21.597080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ad417-df11-44c1-bc97-1011b80ea024 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:21.597371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ad417-df11-44c1-bc97-1011b80ea024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:21.598243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ad417-df11-44c1-bc97-1011b80ea024 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.domain.id=default, user.id=4a4044f3af624daa8f53a39108ebc20d) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:21.598568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ad417-df11-44c1-bc97-1011b80ea024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:21.600077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ad417-df11-44c1-bc97-1011b80ea024 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:21.603415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:21.616735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 32/57] 10.222.0.57 () {56 vars in 1322 bytes} [Fri Jul 15 17:27:21 2022] GET /identity/v3/role_assignments?user.id=4a4044f3af624daa8f53a39108ebc20d&scope.domain.id=default&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:23.569510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83cf5-b31d-4eda-95bf-1baa407c361d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:23.570006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83cf5-b31d-4eda-95bf-1baa407c361d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:23.570093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff83cf5-b31d-4eda-95bf-1baa407c361d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:23.571481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 26/58] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:23.576625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ab12-491e-4c73-988d-79be575e1412 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:23.577029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ab12-491e-4c73-988d-79be575e1412 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:23.577203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ab12-491e-4c73-988d-79be575e1412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:23.594189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-aa95ab12-491e-4c73-988d-79be575e1412 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:23.595390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aa95ab12-491e-4c73-988d-79be575e1412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:23.661598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 33/59] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:23.685746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff83cf5-b31d-4eda-95bf-1baa407c361d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:23.785293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff83cf5-b31d-4eda-95bf-1baa407c361d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:23.807797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:23.818604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11da00-ef0c-472a-b68f-1162c1c4c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:23.819392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11da00-ef0c-472a-b68f-1162c1c4c2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:23.822232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11da00-ef0c-472a-b68f-1162c1c4c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:23.822232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11da00-ef0c-472a-b68f-1162c1c4c2b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:23.822232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11da00-ef0c-472a-b68f-1162c1c4c2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:23.822602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11da00-ef0c-472a-b68f-1162c1c4c2b2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:23.824215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 27/60] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:23.829705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ab12-491e-4c73-988d-79be575e1412 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:23.923106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ab12-491e-4c73-988d-79be575e1412 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:23.950912 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:23.956244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1e492-29e3-4d7d-ae93-915a0cbe170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:23.956950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1e492-29e3-4d7d-ae93-915a0cbe170b None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:23.957247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1e492-29e3-4d7d-ae93-915a0cbe170b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:23.957534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1e492-29e3-4d7d-ae93-915a0cbe170b None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:23.958257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1e492-29e3-4d7d-ae93-915a0cbe170b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:23.959763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1e492-29e3-4d7d-ae93-915a0cbe170b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:24.044773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 34/61] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:27:23 2022] POST /identity/v3/users => generated 275 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:27:27.092775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac789e0-05e2-4807-8968-db8f1ce35f17 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.093820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac789e0-05e2-4807-8968-db8f1ce35f17 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.094314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac789e0-05e2-4807-8968-db8f1ce35f17 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.096013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 28/62] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:27.101456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576921b-678a-471d-8e07-4339a67bbfb9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.102275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576921b-678a-471d-8e07-4339a67bbfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.102762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576921b-678a-471d-8e07-4339a67bbfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.124458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f576921b-678a-471d-8e07-4339a67bbfb9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:27.126104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f576921b-678a-471d-8e07-4339a67bbfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:27.185792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670fabd-a268-4578-84b9-97f39edfbf30 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.186410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670fabd-a268-4578-84b9-97f39edfbf30 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.186915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670fabd-a268-4578-84b9-97f39edfbf30 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.188495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 29/63] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:27.194308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834af42-209f-48be-b45a-7e2c944834cb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.194872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834af42-209f-48be-b45a-7e2c944834cb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.195358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834af42-209f-48be-b45a-7e2c944834cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.221763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1834af42-209f-48be-b45a-7e2c944834cb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:27.223996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1834af42-209f-48be-b45a-7e2c944834cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:27.236132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 35/64] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:27.259800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576921b-678a-471d-8e07-4339a67bbfb9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:27.351211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 30/65] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:27.385648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576921b-678a-471d-8e07-4339a67bbfb9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:27.394952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834af42-209f-48be-b45a-7e2c944834cb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:27.413751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:27.444621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:27.447045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.447045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.447045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.471633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:27.479448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:27.486281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:27.487587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:27.492661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 36/66] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:27.504281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:27.533441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834af42-209f-48be-b45a-7e2c944834cb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:27.559055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:27.578234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d0712-414e-4a7b-a9da-1b8266640faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:27.579053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d0712-414e-4a7b-a9da-1b8266640faa None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.579342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d0712-414e-4a7b-a9da-1b8266640faa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.579623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d0712-414e-4a7b-a9da-1b8266640faa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.597261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d0712-414e-4a7b-a9da-1b8266640faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ba7d0712-414e-4a7b-a9da-1b8266640faa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:27.605383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:27.606365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:27.607131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:27.609388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 31/67] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:27.614090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0b544-dbc5-4a6a-8171-3428d82336f9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.614578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0b544-dbc5-4a6a-8171-3428d82336f9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.615060 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0b544-dbc5-4a6a-8171-3428d82336f9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.616543 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 32/68] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:27.620344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0b544-dbc5-4a6a-8171-3428d82336f9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:27.676248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951441c-d184-48d1-be8a-447f88f9b40c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:27.716129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ddd48-4679-4062-82fb-8726b349903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:27.717271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ddd48-4679-4062-82fb-8726b349903a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.717754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ddd48-4679-4062-82fb-8726b349903a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.718200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ddd48-4679-4062-82fb-8726b349903a None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.719147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ddd48-4679-4062-82fb-8726b349903a None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:27.719742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ddd48-4679-4062-82fb-8726b349903a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:27.721608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ddd48-4679-4062-82fb-8726b349903a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:27.731122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 37/69] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:27.732823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b572-1704-45e4-a320-e78ad127f6f1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.733317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b572-1704-45e4-a320-e78ad127f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.733790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94b572-1704-45e4-a320-e78ad127f6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.757598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1b94b572-1704-45e4-a320-e78ad127f6f1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:27.759172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1b94b572-1704-45e4-a320-e78ad127f6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:27.763097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0b544-dbc5-4a6a-8171-3428d82336f9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:27.792083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:27.802373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:27.803052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.804775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.804775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.804775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:27.804775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:27.806070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:27.815202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 33/70] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:27.816017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:27.843848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 38/71] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:27.843848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3118b85-4991-4ca0-b58f-0c13152a2fce None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.843848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3118b85-4991-4ca0-b58f-0c13152a2fce None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.843848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3118b85-4991-4ca0-b58f-0c13152a2fce None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.843848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 39/72] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:27.843848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3118b85-4991-4ca0-b58f-0c13152a2fce None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:27.878973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:27.934648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cf622-5847-428a-9a45-b490d0dfa4f1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:27.952011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3118b85-4991-4ca0-b58f-0c13152a2fce None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:27.970461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f3592-593f-4ab7-b602-895606d9d320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:27.971847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f3592-593f-4ab7-b602-895606d9d320 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.972137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f3592-593f-4ab7-b602-895606d9d320 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.972411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f3592-593f-4ab7-b602-895606d9d320 None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.985601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:27.986480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.986686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f3592-593f-4ab7-b602-895606d9d320 None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:27.986776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.987695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] PATH_INFO: `/v3/users/4a4044f3af624daa8f53a39108ebc20d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.988943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=4a4044f3af624daa8f53a39108ebc20d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:27.989036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f3592-593f-4ab7-b602-895606d9d320 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:27.990406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:27.990645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 34/73] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:27 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:27.991821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04999d4-1a24-40c6-bc5f-4816b01deb91 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:27.996391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04999d4-1a24-40c6-bc5f-4816b01deb91 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:27.996511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 40/74] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:27 2022] GET /identity/v3/users/4a4044f3af624daa8f53a39108ebc20d => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:27.996615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04999d4-1a24-40c6-bc5f-4816b01deb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:27.996698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.011010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b04999d4-1a24-40c6-bc5f-4816b01deb91 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:28.012196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b04999d4-1a24-40c6-bc5f-4816b01deb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:28.031245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.088875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 35/75] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:28.101204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4768da-668f-45d8-9b9f-d6e2efcf7815 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.101204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4768da-668f-45d8-9b9f-d6e2efcf7815 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.101204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4768da-668f-45d8-9b9f-d6e2efcf7815 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.101204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 36/76] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:28.101204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4768da-668f-45d8-9b9f-d6e2efcf7815 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.101582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037c066-52e3-4a6f-ad2c-193ae4782ca8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.132194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.134686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.135505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.135795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.136074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.146512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:28.154957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:28.155889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:28.156706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 41/77] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:27:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:28.157516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.209948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4768da-668f-45d8-9b9f-d6e2efcf7815 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.243878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a3e53-fdd3-4fc1-b609-51b4e4e20c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.244539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a3e53-fdd3-4fc1-b609-51b4e4e20c92 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.244857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a3e53-fdd3-4fc1-b609-51b4e4e20c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.245424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a3e53-fdd3-4fc1-b609-51b4e4e20c92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.257325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.263070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036a740-1be9-46b9-a10b-218cb71dcaa6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.296957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.297624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.297919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.298208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.312440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a3e53-fdd3-4fc1-b609-51b4e4e20c92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:28.313945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a3e53-fdd3-4fc1-b609-51b4e4e20c92 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:28.343122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.365872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:28.367403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:28.412263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 37/78] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:28.413840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa44265-1236-4e24-96d9-b71b35410f87 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.414140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa44265-1236-4e24-96d9-b71b35410f87 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.414428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa44265-1236-4e24-96d9-b71b35410f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.430016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-aaa44265-1236-4e24-96d9-b71b35410f87 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:28.431213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aaa44265-1236-4e24-96d9-b71b35410f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:28.458504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.465422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 42/79] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:28.466241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.501751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 38/80] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:28.503489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa44265-1236-4e24-96d9-b71b35410f87 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.571495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.572135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c9fe8-7e60-4e4c-82ef-35e24d2587aa None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.605457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.606249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.606536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.606817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.609252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa44265-1236-4e24-96d9-b71b35410f87 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.617040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:28.624688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:28.625727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:28.626529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 43/81] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:28.627162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.646256 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.646930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.647226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.647505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.648264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:28.648576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:28.650008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:28.658249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 39/82] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:28.659073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.689226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.732998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f03ad9-fab3-4dc3-9b94-1131f79aaaa4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.760312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627258f5-23c8-437e-aae6-3c47fd40dfa8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.766214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.767081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.767383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.767667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.769450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:28.770929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:28.772664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 44/83] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:28.773435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.784475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.795042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.798063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.798063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.798063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.798063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:28.799214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:28.800973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 40/84] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:28.801748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.875669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b949626-6d3d-41b8-90ea-c2490002b2a6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.900375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:28.903062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b026dc-ca68-4147-9672-bedf519ae88a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:28.911131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.911939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.912233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.912523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.913759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:28.915272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:28.917129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 45/85] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:28.917718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:28.936884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:28.937563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:28.937858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:28.938142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:28.938901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:28.939379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:28.941393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:28.949583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 41/86] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:28.950578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.004186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:29.022436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68494165-8820-4e8e-a488-76554f55b0bb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.047459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e162c8-9bf4-441c-b064-9a3fc322728e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.056269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:29.057196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:29.057490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:29.057773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:29.058992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:29.060470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:29.062153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 46/87] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:29.062933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.081883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:29.082537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:29.082828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:29.083108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:29.083950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=ec6d14cf644c4cd6b666c86129fcc505) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:29.084270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:29.085744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:29.094197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 42/88] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:28 2022] GET /identity/v3/role_assignments?user.id=ec6d14cf644c4cd6b666c86129fcc505&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:29.094804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.140528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:29.186151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec8b43-4ec0-4a0e-8059-6b38a449de65 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.214776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187eb0fe-c8bf-4267-995a-4c376b5ceb3f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.223992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:29.224705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:29.225004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:29.227842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:29.227842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=4a4044f3af624daa8f53a39108ebc20d) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:29.227842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:29.228113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:29.234852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 47/89] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:29 2022] GET /identity/v3/role_assignments?user.id=4a4044f3af624daa8f53a39108ebc20d&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:29.235537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.243639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:29.268260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:29.269059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:29.269359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:29.269649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:29.270418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:29.271997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:29.366410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494e238-4ce8-47df-9e96-4524347af3f4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.368565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 43/90] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:27:29 2022] POST /identity/v3/users => generated 289 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:29.369902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.411979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:29.429361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:29.430583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:29.431734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:29.432205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:29.434161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:29.436616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:29.439247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 48/91] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:29 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:29.449690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.512085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd93ea5-fc6e-445c-8b64-25f12497e287 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.555677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:29.556460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:29.556845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:29.557197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:29.615446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:29.650851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:29.653795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:29.669352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bbde4-aba2-4460-b8cc-45b51348e4f9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.705174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:29.706374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:29.706667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:29.706951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:29.779624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:29.787125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 44/92] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:29.788341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:29.789922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:29.792707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.922851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:29.927710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 49/93] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:29.933256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:29.955034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc553d-4368-4ecb-8c55-ba1561497787 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:29.999354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:30.000508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:30.001056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:30.001350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:30.003434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:30.005232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:30.007909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 45/94] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:29 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:30.014114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:30.043039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:30.052651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2ae37-b0ce-41ab-9453-ee5f3352a972 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:30.093864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:30.094844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:30.095242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:30.095904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] PATH_INFO: `/v3/projects/6d63fe1a479e4925957c0d117a24db43` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:30.098318 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] RBAC: Authorizing `identity:get_project(project_id=6d63fe1a479e4925957c0d117a24db43)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:30.100018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:30.101813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 50/95] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:29 2022] GET /identity/v3/projects/6d63fe1a479e4925957c0d117a24db43 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:30.107883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:30.151213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dda28-2520-4903-a6ee-07e74b40402f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:30.234584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67f887-2198-4e39-a228-df439735db9f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:30.265905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78a8da-bee5-43f0-9793-311413d25d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:30.266581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78a8da-bee5-43f0-9793-311413d25d55 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:30.266873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78a8da-bee5-43f0-9793-311413d25d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:30.267151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78a8da-bee5-43f0-9793-311413d25d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:30.268746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78a8da-bee5-43f0-9793-311413d25d55 None admin] RBAC: Adding query filter params (role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=ec6d14cf644c4cd6b666c86129fcc505) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:30.269068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78a8da-bee5-43f0-9793-311413d25d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:30.270746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78a8da-bee5-43f0-9793-311413d25d55 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:30.271298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e421-7fa3-4ebe-bf22-973f5d3afea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:30.271964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e421-7fa3-4ebe-bf22-973f5d3afea4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:30.272258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e421-7fa3-4ebe-bf22-973f5d3afea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:30.272678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e421-7fa3-4ebe-bf22-973f5d3afea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:30.273525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e421-7fa3-4ebe-bf22-973f5d3afea4 None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=6d63fe1a479e4925957c0d117a24db43, user.id=ec6d14cf644c4cd6b666c86129fcc505) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:30.273846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e421-7fa3-4ebe-bf22-973f5d3afea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:30.275414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e421-7fa3-4ebe-bf22-973f5d3afea4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:30.282966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 51/96] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:30 2022] GET /identity/v3/role_assignments?user.id=ec6d14cf644c4cd6b666c86129fcc505&scope.project.id=6d63fe1a479e4925957c0d117a24db43&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:30.290745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 46/97] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:30 2022] GET /identity/v3/role_assignments?user.id=ec6d14cf644c4cd6b666c86129fcc505&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:31.545707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7568d-bfed-40da-b844-56fea4eed259 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:31.546208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7568d-bfed-40da-b844-56fea4eed259 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:31.546282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7568d-bfed-40da-b844-56fea4eed259 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:31.547844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 52/98] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:31.566510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e121c-ac5f-45f0-90e1-fc3da63f072b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:31.566924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e121c-ac5f-45f0-90e1-fc3da63f072b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:31.567117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e121c-ac5f-45f0-90e1-fc3da63f072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:31.585151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:31.594142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d54e121c-ac5f-45f0-90e1-fc3da63f072b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:31.599294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d54e121c-ac5f-45f0-90e1-fc3da63f072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:31.684283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 47/99] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:31.713046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7568d-bfed-40da-b844-56fea4eed259 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:31.739106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:31.822652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7568d-bfed-40da-b844-56fea4eed259 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:31.859802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:31.860673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:31.860971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:31.861261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:31.872835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:31.881275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:31.882175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:31.882915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:31.885393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 53/100] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:31.893950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e121c-ac5f-45f0-90e1-fc3da63f072b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:32.016283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e121c-ac5f-45f0-90e1-fc3da63f072b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:32.053185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc77bf5-7536-4473-9440-db98db633676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:32.054263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc77bf5-7536-4473-9440-db98db633676 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.054758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc77bf5-7536-4473-9440-db98db633676 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.055256 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc77bf5-7536-4473-9440-db98db633676 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.056233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc77bf5-7536-4473-9440-db98db633676 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:32.056781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc77bf5-7536-4473-9440-db98db633676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:32.058398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc77bf5-7536-4473-9440-db98db633676 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:32.068016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 48/101] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:32.073673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:32.074979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:32.161746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45cd16-9c74-4f06-b0c8-22966ef1b004 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.162582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45cd16-9c74-4f06-b0c8-22966ef1b004 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.163216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45cd16-9c74-4f06-b0c8-22966ef1b004 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.165700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 49/102] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:32.172156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf602e8-1bc3-449c-8832-b1d5238eda33 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.172940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf602e8-1bc3-449c-8832-b1d5238eda33 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.173440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf602e8-1bc3-449c-8832-b1d5238eda33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.206221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-edf602e8-1bc3-449c-8832-b1d5238eda33 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:32.208470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-edf602e8-1bc3-449c-8832-b1d5238eda33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:32.247403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:32.277802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26ef2-6e50-4e00-9ddb-0107d1056dba None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:32.327918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 50/103] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:32.343959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:32.345249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.345757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.346275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.347734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:32.349527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:32.351442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 54/104] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:32 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:32.357257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf602e8-1bc3-449c-8832-b1d5238eda33 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:32.415774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:32.536731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf602e8-1bc3-449c-8832-b1d5238eda33 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:32.579852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82b0ec-a426-42f5-ac50-1c7f81fc5e52 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:32.613647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:32.637337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:32.638326 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.638632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.638917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.639850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:32.641005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.641506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.641984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.646648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:32.648891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:32.654872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 51/105] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:32 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:32.661131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:32.661358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:32.670630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:32.671563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:32.672330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 55/106] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:32.682365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:32.717935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:32.841200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6d63-a56e-4e61-b04a-96e54375598c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:32.895725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc687b5-ecf3-4456-8184-b51833b3a854 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:32.932936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93da3c-bf9a-4e2f-b1ef-410d965d10d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:32.933674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93da3c-bf9a-4e2f-b1ef-410d965d10d2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.933966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93da3c-bf9a-4e2f-b1ef-410d965d10d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.934299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93da3c-bf9a-4e2f-b1ef-410d965d10d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.935068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93da3c-bf9a-4e2f-b1ef-410d965d10d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:32.935380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93da3c-bf9a-4e2f-b1ef-410d965d10d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:32.936814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93da3c-bf9a-4e2f-b1ef-410d965d10d2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:32.938200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf8d9d-22d2-4aca-ba55-749d2bb1660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:32.939103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf8d9d-22d2-4aca-ba55-749d2bb1660c None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.939392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf8d9d-22d2-4aca-ba55-749d2bb1660c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.939671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf8d9d-22d2-4aca-ba55-749d2bb1660c None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/ec6d14cf644c4cd6b666c86129fcc505/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.942271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf8d9d-22d2-4aca-ba55-749d2bb1660c None admin] RBAC: Authorizing `identity:create_grant(project_id=897b8b7b9c0b4449800d389268bf55b0, user_id=ec6d14cf644c4cd6b666c86129fcc505, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:32.943715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf8d9d-22d2-4aca-ba55-749d2bb1660c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:32.946375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 56/107] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:32.947784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7728a-1f8a-4c00-bc01-66c861f168b1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:32.948079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7728a-1f8a-4c00-bc01-66c861f168b1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:32.948367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7728a-1f8a-4c00-bc01-66c861f168b1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:32.949903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 57/108] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:32.954294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7728a-1f8a-4c00-bc01-66c861f168b1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:33.037029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:33.040751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 52/109] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:32 2022] PUT /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/ec6d14cf644c4cd6b666c86129fcc505/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:33.042042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f1c64-ff50-46dc-8bd2-b2ed2cecf642 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.042335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f1c64-ff50-46dc-8bd2-b2ed2cecf642 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.042619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f1c64-ff50-46dc-8bd2-b2ed2cecf642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.066553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7728a-1f8a-4c00-bc01-66c861f168b1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:33.078545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-054f1c64-ff50-46dc-8bd2-b2ed2cecf642 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:33.079773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-054f1c64-ff50-46dc-8bd2-b2ed2cecf642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:33.105440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:33.106259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.106882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.107173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] PATH_INFO: `/v3/users/4a4044f3af624daa8f53a39108ebc20d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.108447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] RBAC: Authorizing `identity:get_user(user_id=4a4044f3af624daa8f53a39108ebc20d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:33.110128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:33.112062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 58/110] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:32 2022] GET /identity/v3/users/4a4044f3af624daa8f53a39108ebc20d => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:33.117876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:33.215945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:33.234856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41b469-ba4e-470e-a760-942e633728c3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:33.254810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 53/111] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:33.287195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac93570-3a45-4faf-ba7d-7b9031499bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:33.288103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac93570-3a45-4faf-ba7d-7b9031499bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.288396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac93570-3a45-4faf-ba7d-7b9031499bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.288703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac93570-3a45-4faf-ba7d-7b9031499bd4 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.289605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba4633-189f-491f-af3c-8271c48cc55a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.290104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba4633-189f-491f-af3c-8271c48cc55a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.290587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba4633-189f-491f-af3c-8271c48cc55a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.291863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac93570-3a45-4faf-ba7d-7b9031499bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:33.292587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 54/112] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:33.294609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac93570-3a45-4faf-ba7d-7b9031499bd4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:33.295458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 59/113] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:33 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:33.301632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba4633-189f-491f-af3c-8271c48cc55a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:33.307101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff476c-43d6-41f1-9eb1-b9bfd807a24d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.307397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff476c-43d6-41f1-9eb1-b9bfd807a24d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.307751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff476c-43d6-41f1-9eb1-b9bfd807a24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.347323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a7ff476c-43d6-41f1-9eb1-b9bfd807a24d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:33.348552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a7ff476c-43d6-41f1-9eb1-b9bfd807a24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:33.352613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:33.432938 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 60/114] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:33.433786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff476c-43d6-41f1-9eb1-b9bfd807a24d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:33.467529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba4633-189f-491f-af3c-8271c48cc55a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:33.536156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d6ef1-947d-4472-b92f-7eaee1ce9f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:33.537093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d6ef1-947d-4472-b92f-7eaee1ce9f6e None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.537390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d6ef1-947d-4472-b92f-7eaee1ce9f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.537671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d6ef1-947d-4472-b92f-7eaee1ce9f6e None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/4a4044f3af624daa8f53a39108ebc20d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.539733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d6ef1-947d-4472-b92f-7eaee1ce9f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=897b8b7b9c0b4449800d389268bf55b0, user_id=4a4044f3af624daa8f53a39108ebc20d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:33.541249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d6ef1-947d-4472-b92f-7eaee1ce9f6e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:33.582760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff476c-43d6-41f1-9eb1-b9bfd807a24d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:33.617796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:33.618599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.618955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.619173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.626672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:33.629645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:33.637370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:33.641736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:33.642503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 61/115] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:33.643643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 55/116] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:33 2022] PUT /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/4a4044f3af624daa8f53a39108ebc20d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:33.644054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:33.644125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f291c-141b-40f2-a9ad-f3cf447e71ac None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.644125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f291c-141b-40f2-a9ad-f3cf447e71ac None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.644125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f291c-141b-40f2-a9ad-f3cf447e71ac None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.654258 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 56/117] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:33 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:33.655945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d34544-6fa8-47a1-860a-575b53c20127 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.656457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d34544-6fa8-47a1-860a-575b53c20127 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.656961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d34544-6fa8-47a1-860a-575b53c20127 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.658418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 57/118] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:33.669769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d34544-6fa8-47a1-860a-575b53c20127 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:33.737808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:33.772209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2746af-102b-4010-a230-ae65b4590ff0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:33.813890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d34544-6fa8-47a1-860a-575b53c20127 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:33.815362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65eab0-8a07-494f-8806-7694a6170c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:33.816492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65eab0-8a07-494f-8806-7694a6170c14 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.816985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65eab0-8a07-494f-8806-7694a6170c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.817528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65eab0-8a07-494f-8806-7694a6170c14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.834280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65eab0-8a07-494f-8806-7694a6170c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-aa65eab0-8a07-494f-8806-7694a6170c14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:33.843113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:33.844014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:33.844797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 62/119] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:33.846441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f7a6-9a1e-497d-a98d-4d1e3aacdc00 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.846441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f7a6-9a1e-497d-a98d-4d1e3aacdc00 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.846689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f7a6-9a1e-497d-a98d-4d1e3aacdc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.867653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf97cd6-c468-41e8-bfd3-8c8ecfc1958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:33.868333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97cd6-c468-41e8-bfd3-8c8ecfc1958c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.868639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97cd6-c468-41e8-bfd3-8c8ecfc1958c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.868920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97cd6-c468-41e8-bfd3-8c8ecfc1958c None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.869682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf97cd6-c468-41e8-bfd3-8c8ecfc1958c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:33.869990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf97cd6-c468-41e8-bfd3-8c8ecfc1958c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:33.870406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7619f7a6-9a1e-497d-a98d-4d1e3aacdc00 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:33.871392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf97cd6-c468-41e8-bfd3-8c8ecfc1958c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:33.872099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7619f7a6-9a1e-497d-a98d-4d1e3aacdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:33.881141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 58/120] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:33.882204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c7e35-5444-4fe9-aefc-0237c9a0c3cd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:33.882496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c7e35-5444-4fe9-aefc-0237c9a0c3cd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:33.882769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c7e35-5444-4fe9-aefc-0237c9a0c3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:33.906656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2f6c7e35-5444-4fe9-aefc-0237c9a0c3cd None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:33.908321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2f6c7e35-5444-4fe9-aefc-0237c9a0c3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:33.952893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:33.966233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 63/121] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:33.976775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f7a6-9a1e-497d-a98d-4d1e3aacdc00 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.006825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 59/122] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:34.007505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c7e35-5444-4fe9-aefc-0237c9a0c3cd None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.061245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:34.123382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f7a6-9a1e-497d-a98d-4d1e3aacdc00 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.126074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c7e35-5444-4fe9-aefc-0237c9a0c3cd None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.167590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.168269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.168563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.168875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.169633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:34.170013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.170690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.171364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.171712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.172072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.172427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] PATH_INFO: `/v3/users/607819d7e6bf4f4b85abdef3465251d4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.180624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 64/123] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:34.181285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.187362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] RBAC: Authorizing `identity:get_user(user_id=607819d7e6bf4f4b85abdef3465251d4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.188901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.195826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 60/124] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/users/607819d7e6bf4f4b85abdef3465251d4 => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:34.196625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.235533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:34.286956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836540-361c-4a22-9f20-0b213c11d058 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.295571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:34.302647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b475d-19f4-42a6-a181-8e8c569e22c3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.323798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.324922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.325357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.325782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.337025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:34.339019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.339824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.340118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.340398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.341646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.343102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.344846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 61/125] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:34.345624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:34.346651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:34.355830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 65/126] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:34.356591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.460360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989a44c-a0fe-4833-84ea-72953a4470e4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.464954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba17dcf-72f4-4318-a406-cacffeebe668 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.480022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:34.497168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.499649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.499649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.499649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.499945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.501419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.503279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 62/127] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:34.506015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.506976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.507391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.507679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.508165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.541318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:34.584410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.586675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.612191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909ff45-1d5d-462b-b702-025ce70e645f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.650505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.652831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.653448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.654012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.655214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:34.656102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.658122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.671808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 63/128] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:34.673315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.701365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 66/129] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:34.702900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.723145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:34.784800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:34.791802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f043c-eaf0-4a6d-b2e5-c637f87778cc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.816752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925386f-fe90-4821-a349-cf173d98a950 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:34.836359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.837184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.837568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.837907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.839028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.841228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.859462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:34.860887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:34.861401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:34.862538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] PATH_INFO: `/v3/projects/6d63fe1a479e4925957c0d117a24db43` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:34.863870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] RBAC: Authorizing `identity:get_project(project_id=6d63fe1a479e4925957c0d117a24db43)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:34.865501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:34.867518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 67/130] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/projects/6d63fe1a479e4925957c0d117a24db43 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:34.868247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.955590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 64/131] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:27:34 2022] POST /identity/v3/projects => generated 302 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:34.956965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:34.968483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:35.014848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cba058-4d07-4aaf-b819-e6202365b251 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:35.090202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab38a3-2eeb-4bd4-829e-0ba9573d86b0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:35.121442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:35.122491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.122785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.123074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.124416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:35.125916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:35.128578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 68/132] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:35.129307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:35.137222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:35.138111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.138397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.138674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.140823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:35.142320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:35.145012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 65/133] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:34 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:35.145968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:35.268635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:35.274421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e62f2-9bc7-495f-b38a-e3c3b1cfe8f5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:35.280803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eab24b-82c0-4377-92cd-34ff563cef43 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:35.322513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:35.323596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.323888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.324188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] PATH_INFO: `/v3/projects/6d63fe1a479e4925957c0d117a24db43/users/ec6d14cf644c4cd6b666c86129fcc505/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.324984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:35.325967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.326362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.326747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.326910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d63fe1a479e4925957c0d117a24db43, user_id=ec6d14cf644c4cd6b666c86129fcc505, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:35.328483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:35.328897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:35.334110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:35.336196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 66/134] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:35 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:35.340932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:35.430061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:35.451473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 69/135] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:35 2022] PUT /identity/v3/projects/6d63fe1a479e4925957c0d117a24db43/users/ec6d14cf644c4cd6b666c86129fcc505/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:35.452756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:35.457097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07057af7-58cd-466a-8df7-abad9fd35c88 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:35.468662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:35.508730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae60897c-1282-4654-9182-e07739e2b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:35.510135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60897c-1282-4654-9182-e07739e2b726 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.510437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60897c-1282-4654-9182-e07739e2b726 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.510725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60897c-1282-4654-9182-e07739e2b726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.511774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60897c-1282-4654-9182-e07739e2b726 None admin] RBAC: Adding query filter params (role.id=2e646c3c6fd546cda41230bb92e3df1d, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=607819d7e6bf4f4b85abdef3465251d4) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:35.512099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60897c-1282-4654-9182-e07739e2b726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:35.514178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae60897c-1282-4654-9182-e07739e2b726 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:35.522542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 67/136] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:35 2022] GET /identity/v3/role_assignments?user.id=607819d7e6bf4f4b85abdef3465251d4&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=2e646c3c6fd546cda41230bb92e3df1d => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:35.523855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ed34-4744-416e-9d69-58bd8103f6d2 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.524212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ed34-4744-416e-9d69-58bd8103f6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.524505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6ed34-4744-416e-9d69-58bd8103f6d2 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.526338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 68/137] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:35.540439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec99cc-494f-46c2-8a45-284c27497ba9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.540813 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec99cc-494f-46c2-8a45-284c27497ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.541109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec99cc-494f-46c2-8a45-284c27497ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.578866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-53ec99cc-494f-46c2-8a45-284c27497ba9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:35.580199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-53ec99cc-494f-46c2-8a45-284c27497ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:35.680177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c7d89-e01e-41bc-a0dd-45caa3c57f12 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:35.752281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 69/138] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:35.774040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5b678-ca57-4e9a-ab51-73a6329c9390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:35.774960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5b678-ca57-4e9a-ab51-73a6329c9390 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.775262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5b678-ca57-4e9a-ab51-73a6329c9390 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.775544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5b678-ca57-4e9a-ab51-73a6329c9390 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/ec6d14cf644c4cd6b666c86129fcc505/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.778833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5b678-ca57-4e9a-ab51-73a6329c9390 None admin] RBAC: Authorizing `identity:create_grant(project_id=897b8b7b9c0b4449800d389268bf55b0, user_id=ec6d14cf644c4cd6b666c86129fcc505, role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:35.780347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5b678-ca57-4e9a-ab51-73a6329c9390 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:35.810030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec99cc-494f-46c2-8a45-284c27497ba9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:35.862898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 70/139] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:35 2022] PUT /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/ec6d14cf644c4cd6b666c86129fcc505/roles/d74c967ed68244238636f7d9bb0517c6 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:35.918198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:35.949937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec99cc-494f-46c2-8a45-284c27497ba9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:35.961260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a680-0a2e-46b3-a39c-55bd8ee5323a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.961479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a680-0a2e-46b3-a39c-55bd8ee5323a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.961745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a680-0a2e-46b3-a39c-55bd8ee5323a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:35.963465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 71/140] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:35.974323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df0898-64c5-4f44-9b17-6c0f8f2da3fb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:35.974615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df0898-64c5-4f44-9b17-6c0f8f2da3fb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:35.974900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df0898-64c5-4f44-9b17-6c0f8f2da3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:36.004879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-65df0898-64c5-4f44-9b17-6c0f8f2da3fb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:36.006094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-65df0898-64c5-4f44-9b17-6c0f8f2da3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:36.014408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:36.015604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:36.016115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:36.016589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:36.040586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:36.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:36.054123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:36.054878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:36.057318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 70/141] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:36.063594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:36.166306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:36.181984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 72/142] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:36.213508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df0898-64c5-4f44-9b17-6c0f8f2da3fb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:36.266041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3439-da4b-4b5d-a643-39496d57bed6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:36.335170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:36.335857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:36.336150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:36.336435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:36.337238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:36.337557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:36.338964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:36.348667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:36.373102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 71/143] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:36.378025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:36.423220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df0898-64c5-4f44-9b17-6c0f8f2da3fb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:36.487142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:36.488036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:36.488330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:36.488623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:36.509970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:36.518906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:36.519779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:36.520524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 73/144] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:36.530293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:36.622849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffd2ad-b0e4-4a20-9a56-3323ae6c17f0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:36.647467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:36.697143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177951b-dfb8-4ff6-8df7-c3326dd38d49 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:36.717361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:36.718270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:36.718562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:36.718844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:36.724636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:36.724636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:36.724636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 72/145] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:36 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:36.729284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:36.761961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:36.762988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:36.763574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:36.764055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:36.765058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:36.765564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:36.767175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:36.777189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 74/146] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:36.783333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:36.898636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:36.913235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3572db-89fe-4efd-a309-1b0ca5ed19a8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:36.944704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:36.970756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:36.971682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:36.971973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:36.972250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] PATH_INFO: `/v3/users/4a4044f3af624daa8f53a39108ebc20d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:36.977376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] RBAC: Authorizing `identity:get_user(user_id=4a4044f3af624daa8f53a39108ebc20d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:36.978845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:36.980537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 75/147] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:36 2022] GET /identity/v3/users/4a4044f3af624daa8f53a39108ebc20d => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:36.986135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:36.989412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814290d-99e1-498c-8c53-77af62c18669 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:37.057335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:37.058009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:37.058305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:37.058589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:37.176908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:37.178784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:37.216945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8716f-9166-4f5d-963b-3d039c36c92b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:37.305237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:37.306031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:37.306322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:37.306602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:37.345093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:37.404462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:37.421895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:37.423661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:37.429101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 73/148] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:27:37.435303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:37.608375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 76/149] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:27:37.624746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298828-ff67-4685-b30c-473cced02326 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:37.634069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:37.676729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:37.677614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:37.677905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:37.678191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:37.679419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:37.680922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:37.682665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 74/150] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:37 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:37.688753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:37.741915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:37.807831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:37.831781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a92a79-3ab4-4fd3-b281-93cb5f8e1821 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:37.838996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158a127-4e55-48af-8dd1-125c25a03910 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:37.895556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:37.896507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:37.896919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:37.897207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:37.901338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:37.903117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:37.904964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 77/151] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:37 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:37.906356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dbaf6-1b6b-4037-8374-db3d4091f4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:37.907037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dbaf6-1b6b-4037-8374-db3d4091f4dc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:37.907328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dbaf6-1b6b-4037-8374-db3d4091f4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:37.907612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dbaf6-1b6b-4037-8374-db3d4091f4dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:37.908529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dbaf6-1b6b-4037-8374-db3d4091f4dc None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=ec6d14cf644c4cd6b666c86129fcc505) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:37.909378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dbaf6-1b6b-4037-8374-db3d4091f4dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:37.910834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dbaf6-1b6b-4037-8374-db3d4091f4dc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:37.921604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:37.924568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 75/152] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:37 2022] GET /identity/v3/role_assignments?user.id=ec6d14cf644c4cd6b666c86129fcc505&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:38.106781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0ac8c-c160-49cf-96fd-ebba94a3fd5c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:38.124229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0f3d1-0d85-4329-88d6-7963688df79b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.124557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0f3d1-0d85-4329-88d6-7963688df79b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.124853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0f3d1-0d85-4329-88d6-7963688df79b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.126255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 76/153] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:38.134368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c959f76-e89a-4048-941d-d0a778204d70 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.134665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c959f76-e89a-4048-941d-d0a778204d70 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.134954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c959f76-e89a-4048-941d-d0a778204d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.149012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:38.150009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.150471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.150969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.152289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=4a4044f3af624daa8f53a39108ebc20d) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:38.152903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:38.154623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:38.162351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 78/154] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:37 2022] GET /identity/v3/role_assignments?user.id=4a4044f3af624daa8f53a39108ebc20d&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:38.168653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0c959f76-e89a-4048-941d-d0a778204d70 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:38.168653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0c959f76-e89a-4048-941d-d0a778204d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:38.299479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 77/155] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:38.336313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:38.366426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:38.464474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc19f6-adc0-4475-ba98-2fc389c312d5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:38.510976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:38.512137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.512762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.513523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.514902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b640-0c4d-438f-8fb9-62a551b51ff4 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.515203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b640-0c4d-438f-8fb9-62a551b51ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.515489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b640-0c4d-438f-8fb9-62a551b51ff4 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.515761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:38.517063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 78/156] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:38.518507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:38.520391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 79/157] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:38.524642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca455-a231-43bb-a1a3-2a1ee5092602 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.525182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca455-a231-43bb-a1a3-2a1ee5092602 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.525496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca455-a231-43bb-a1a3-2a1ee5092602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.526367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:38.551727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-42fca455-a231-43bb-a1a3-2a1ee5092602 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:38.552992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-42fca455-a231-43bb-a1a3-2a1ee5092602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:38.587505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:38.635750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:38.644710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 79/158] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:38.644710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ebbe2-ccf4-432c-b6f0-d4c964b3aa0d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.644710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ebbe2-ccf4-432c-b6f0-d4c964b3aa0d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.644710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ebbe2-ccf4-432c-b6f0-d4c964b3aa0d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.646299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 80/159] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:38.650656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187e864-261a-4d80-b2e0-532815b3fcdf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:38.651459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6eab-3b7b-4595-aeb1-89f2016f29a4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.652252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6eab-3b7b-4595-aeb1-89f2016f29a4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.652354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df6eab-3b7b-4595-aeb1-89f2016f29a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.670778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c9df6eab-3b7b-4595-aeb1-89f2016f29a4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:38.671962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c9df6eab-3b7b-4595-aeb1-89f2016f29a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:38.691292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:38.691979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.692265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.692546 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.693298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:38.694766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:38.744534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 81/160] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:38.745683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df6eab-3b7b-4595-aeb1-89f2016f29a4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:38.803743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 80/161] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:27:38 2022] POST /identity/v3/users => generated 283 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:27:38.804743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:38.851390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:38.867587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df6eab-3b7b-4595-aeb1-89f2016f29a4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:38.910796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03284a08-9774-429b-a904-8e8e3ba3c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:38.911669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03284a08-9774-429b-a904-8e8e3ba3c904 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.911964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03284a08-9774-429b-a904-8e8e3ba3c904 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.912243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03284a08-9774-429b-a904-8e8e3ba3c904 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.925378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03284a08-9774-429b-a904-8e8e3ba3c904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:38.925953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13dce-4b93-44b8-b2e2-5e670d170484 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-03284a08-9774-429b-a904-8e8e3ba3c904 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:38.934079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:38.935048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:38.935801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 82/162] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:38.942185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3dc92-e834-48d6-92e9-34e02fd9b90e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.942482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3dc92-e834-48d6-92e9-34e02fd9b90e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.942790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3dc92-e834-48d6-92e9-34e02fd9b90e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.944223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 83/163] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:38.949183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3dc92-e834-48d6-92e9-34e02fd9b90e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:38.971433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee23788-ab44-4f29-8967-a8d2f77044ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:38.972249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee23788-ab44-4f29-8967-a8d2f77044ff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.972589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee23788-ab44-4f29-8967-a8d2f77044ff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.972900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee23788-ab44-4f29-8967-a8d2f77044ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:38.985875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee23788-ab44-4f29-8967-a8d2f77044ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4ee23788-ab44-4f29-8967-a8d2f77044ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:38.994628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:38.995471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:38.996257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 81/164] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:38.998174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b484a3-c019-4a60-98c2-591da913e526 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:38.998470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b484a3-c019-4a60-98c2-591da913e526 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:38.999259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b484a3-c019-4a60-98c2-591da913e526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.008893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:39.019682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-01b484a3-c019-4a60-98c2-591da913e526 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:39.022110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-01b484a3-c019-4a60-98c2-591da913e526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:39.077996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3dc92-e834-48d6-92e9-34e02fd9b90e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.120370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 82/165] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:39.120370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b484a3-c019-4a60-98c2-591da913e526 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.138805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.139550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.139826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.140100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.140886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:39.141191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:39.142582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:39.153644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 84/166] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:39.156698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.173114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:39.242679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b484a3-c019-4a60-98c2-591da913e526 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.286288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.286983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.287289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.287815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.288589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:39.292683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:39.292683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:39.293604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:39.296457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388d571-e454-4325-957c-69e41f04b035 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.302749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 83/167] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:39.303365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.347588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.348414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.348733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.349154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.362582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:39.370839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:39.374001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 85/168] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:39.374848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.457646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91cae1-5eee-4cef-844d-39f1a955a08b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.460977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:39.483745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd021-36ff-4727-af7a-c3f21d49b713 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.503248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.507303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.507627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.508984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] PATH_INFO: `/v3/users/607819d7e6bf4f4b85abdef3465251d4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.510697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] RBAC: Authorizing `identity:get_user(user_id=607819d7e6bf4f4b85abdef3465251d4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:39.512812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:39.515671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 84/169] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/users/607819d7e6bf4f4b85abdef3465251d4 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:39.516868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.524248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.525396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.525780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.526154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.527844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:39.529501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:39.531289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 86/170] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:39.532145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.608097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:39.653997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4740b2-2ee7-422b-8f4f-c3af718bee1b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.675429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2dee5-d89a-4f6a-be99-bd60774d2d1e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.719313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.720239 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.720519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.720859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.721614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:39.721920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:39.723309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:39.732585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 85/171] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:39.733452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.761435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.762240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.762525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.762803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.764112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:39.765596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:39.767633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 87/172] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:39.769037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:39.790108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:39.856877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58edc3-8512-4e81-82e2-bc9314382812 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.890404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:39.899817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:39.900484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:39.900802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:39.901077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:39.950346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb9a4f-6dcb-444f-bc09-5954e33dd0c1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:39.991562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:39.993076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:40.013351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:40.014300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:40.014680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:40.015049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] PATH_INFO: `/v3/users/ec6d14cf644c4cd6b666c86129fcc505` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:40.016361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] RBAC: Authorizing `identity:get_user(user_id=ec6d14cf644c4cd6b666c86129fcc505)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:40.017983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:40.020878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 88/173] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/users/ec6d14cf644c4cd6b666c86129fcc505 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:40.021628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:40.143124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:40.171233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02044cb-4ff2-4bb2-80dd-784d4f4eb689 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:40.208771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:40.210072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:40.210511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:40.210795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/607819d7e6bf4f4b85abdef3465251d4/roles/2e646c3c6fd546cda41230bb92e3df1d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:40.212977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] RBAC: Authorizing `identity:create_grant(project_id=897b8b7b9c0b4449800d389268bf55b0, user_id=607819d7e6bf4f4b85abdef3465251d4, role_id=2e646c3c6fd546cda41230bb92e3df1d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:40.214445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:40.226392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 86/174] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:40.227262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:40.263955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:40.322429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 89/175] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:40 2022] PUT /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0/users/607819d7e6bf4f4b85abdef3465251d4/roles/2e646c3c6fd546cda41230bb92e3df1d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:40.323312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:40.393634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93048442-365a-49e7-af28-5aac892f19d0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:40.450877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:40.451944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:40.451944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:40.452113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:40.515562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb346-87d2-4cd4-a355-3cfb1e1fd038 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:40.527823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:40.529741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:40.555299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:40.557111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:40.557786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:40.558347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] PATH_INFO: `/v3/projects/6d63fe1a479e4925957c0d117a24db43` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:40.559491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:40.561485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] RBAC: Authorizing `identity:get_project(project_id=6d63fe1a479e4925957c0d117a24db43)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:40.563173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:40.566908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 90/176] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:40 2022] GET /identity/v3/projects/6d63fe1a479e4925957c0d117a24db43 => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:40.578828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:40.584720 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:40.670242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 87/177] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:40.678214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:40.815045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf49ce-e3dd-4461-b574-37255e4b8f7e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:40.820512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84285c37-6b81-4684-b6d7-89d45272719b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:40.857899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:40.871923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:40.873184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:40.873641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:40.877585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:40.878442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:40.880201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:40.882257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 88/178] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:40 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:40.886302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ddbd9-72b2-4b99-9b51-d94628239127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:40.886977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ddbd9-72b2-4b99-9b51-d94628239127 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:40.887261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ddbd9-72b2-4b99-9b51-d94628239127 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:40.887663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ddbd9-72b2-4b99-9b51-d94628239127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:40.888372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ddbd9-72b2-4b99-9b51-d94628239127 None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=6d63fe1a479e4925957c0d117a24db43, user.id=ec6d14cf644c4cd6b666c86129fcc505) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:40.888706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ddbd9-72b2-4b99-9b51-d94628239127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:40.889101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:40.890154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ddbd9-72b2-4b99-9b51-d94628239127 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:40.903177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 91/179] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:40 2022] GET /identity/v3/role_assignments?user.id=ec6d14cf644c4cd6b666c86129fcc505&scope.project.id=6d63fe1a479e4925957c0d117a24db43&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:41.051650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ecb5f-fca8-4acb-9e1f-b3a94332db2e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:41.103821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6276b-2f5a-4974-a361-151e2784ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:41.104834 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6276b-2f5a-4974-a361-151e2784ec32 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.105305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6276b-2f5a-4974-a361-151e2784ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.105763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6276b-2f5a-4974-a361-151e2784ec32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.106871 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6276b-2f5a-4974-a361-151e2784ec32 None admin] RBAC: Adding query filter params (role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=ec6d14cf644c4cd6b666c86129fcc505) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:41.107374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6276b-2f5a-4974-a361-151e2784ec32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:41.109170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6276b-2f5a-4974-a361-151e2784ec32 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:41.119622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 89/180] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:40 2022] GET /identity/v3/role_assignments?user.id=ec6d14cf644c4cd6b666c86129fcc505&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:41.522251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7209075-2023-4154-805d-8b4db47b29fa None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.523059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7209075-2023-4154-805d-8b4db47b29fa None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.523494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7209075-2023-4154-805d-8b4db47b29fa None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.525047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 92/181] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:41.530795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07927a3f-bc5b-430d-8bc8-53b5992da07e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.531284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07927a3f-bc5b-430d-8bc8-53b5992da07e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.531705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07927a3f-bc5b-430d-8bc8-53b5992da07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.545544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cc49b-9d43-4eca-b872-875e4bf1a45a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.545841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cc49b-9d43-4eca-b872-875e4bf1a45a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.546093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cc49b-9d43-4eca-b872-875e4bf1a45a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.547590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 93/182] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:41.553030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa563a22-c2c9-40b2-8293-9e28d3f9d3b2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.553304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa563a22-c2c9-40b2-8293-9e28d3f9d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.553579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa563a22-c2c9-40b2-8293-9e28d3f9d3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.556449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-07927a3f-bc5b-430d-8bc8-53b5992da07e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:41.557703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-07927a3f-bc5b-430d-8bc8-53b5992da07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:41.580794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-aa563a22-c2c9-40b2-8293-9e28d3f9d3b2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:41.582033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aa563a22-c2c9-40b2-8293-9e28d3f9d3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:41.632388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:41.653457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 90/183] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:41.679069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:41.686970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07927a3f-bc5b-430d-8bc8-53b5992da07e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:41.691968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 94/184] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:41.719838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa563a22-c2c9-40b2-8293-9e28d3f9d3b2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:41.813990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07927a3f-bc5b-430d-8bc8-53b5992da07e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:41.851770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:41.852927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.853413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.853885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.858930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa563a22-c2c9-40b2-8293-9e28d3f9d3b2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:41.866097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:41.874295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:41.875177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:41.875980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:41.878399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 91/185] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:41.884110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:41.899570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b2014-d0bf-43a2-8fee-4a02b4236a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:41.900389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b2014-d0bf-43a2-8fee-4a02b4236a8a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.900703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b2014-d0bf-43a2-8fee-4a02b4236a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.900983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b2014-d0bf-43a2-8fee-4a02b4236a8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.916916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b2014-d0bf-43a2-8fee-4a02b4236a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3b9b2014-d0bf-43a2-8fee-4a02b4236a8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:41.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:41.926852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:41.927636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 95/186] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:27:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:41.933171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d738c32-b14d-4c67-8f33-3f81190ba524 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:41.933466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d738c32-b14d-4c67-8f33-3f81190ba524 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:41.933748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d738c32-b14d-4c67-8f33-3f81190ba524 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:41.935267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 96/187] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:41.935816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d738c32-b14d-4c67-8f33-3f81190ba524 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:41.947814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:41.998104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76177e0-2a95-4c6b-a54e-c4ce8521cc5f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.007863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:42.034236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf07cd-413d-4b70-a2dc-9349193026eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.034907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf07cd-413d-4b70-a2dc-9349193026eb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.035191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf07cd-413d-4b70-a2dc-9349193026eb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.035468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf07cd-413d-4b70-a2dc-9349193026eb None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.036213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf07cd-413d-4b70-a2dc-9349193026eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:42.036517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf07cd-413d-4b70-a2dc-9349193026eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.037939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf07cd-413d-4b70-a2dc-9349193026eb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.046245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 92/188] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:42.047271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01b14b-6fb5-4044-ad20-9a7229664a24 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.047561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01b14b-6fb5-4044-ad20-9a7229664a24 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.047838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01b14b-6fb5-4044-ad20-9a7229664a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.061145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d738c32-b14d-4c67-8f33-3f81190ba524 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.063533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ff01b14b-6fb5-4044-ad20-9a7229664a24 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:42.065675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ff01b14b-6fb5-4044-ad20-9a7229664a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:42.098184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.098868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.099155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.099436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.100187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:42.100494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.101932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.111356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 97/189] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:27:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:42.111811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.138588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 93/190] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:42.139561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01b14b-6fb5-4044-ad20-9a7229664a24 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.191983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:42.230485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff3491-3aea-4cf1-92e9-c7f55095bb1b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.251421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:42.261218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01b14b-6fb5-4044-ad20-9a7229664a24 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.274822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.275778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.276067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.276345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] PATH_INFO: `/v3/users/3d304d9729a146c6a572afe56c525132` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.291949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] RBAC: Authorizing `identity:get_user(user_id=3d304d9729a146c6a572afe56c525132)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.294604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.305626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 98/191] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/users/3d304d9729a146c6a572afe56c525132 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:42.305626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.310606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.310606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.310863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.311092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] PATH_INFO: `/v3/users/3d304d9729a146c6a572afe56c525132` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.312299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] RBAC: Authorizing `identity:get_user(user_id=3d304d9729a146c6a572afe56c525132)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.313935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.316463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 94/192] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/users/3d304d9729a146c6a572afe56c525132 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:42.317046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.415454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605abb1-e7c8-49c1-b4f3-1f3092070a86 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.421457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686c1a1-2577-4ebb-a36d-b5660d718f8d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.431207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:42.452424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.453448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.453737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.454019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.455249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.456774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.457630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.458291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.458721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 99/193] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:42.459498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.463158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.463470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.510375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:42.535646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.537186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.575881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361ce90-c932-4bad-8daa-c4a9a4f5e3d9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.619596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.620248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.620534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.620840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.649040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 95/194] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:42.654931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.665893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:42.696361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.697872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.753128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:42.757798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66ed47-af9b-4090-a1b0-2cfc08ce030c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.794950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.795736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.796025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.796302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.797158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.798649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.813257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 100/195] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:42.818194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.912147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 96/196] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:27:42 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:42.913262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.917817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0547b78-3c51-4ac4-b0e5-6a5dbe7e8157 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:42.961156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:42.962326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:42.962801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:42.963245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:42.964648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:42.966349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:42.968694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 101/197] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:42.974981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:42.992904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:43.012478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:43.062719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c029b-cccc-4a20-a498-5d97ec9a0e8a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:43.101602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8386e-46c5-455b-8385-55064bbbc335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:43.102496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8386e-46c5-455b-8385-55064bbbc335 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.102680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad7af1-033f-48fc-a528-abf624768c8a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:43.102801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8386e-46c5-455b-8385-55064bbbc335 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.103054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8386e-46c5-455b-8385-55064bbbc335 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.104312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8386e-46c5-455b-8385-55064bbbc335 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:43.105831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8386e-46c5-455b-8385-55064bbbc335 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:43.107715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 97/198] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:43.108878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df1804-53c4-4c44-9e5e-78cf6f84106b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.109167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df1804-53c4-4c44-9e5e-78cf6f84106b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.109450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df1804-53c4-4c44-9e5e-78cf6f84106b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.115101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 98/199] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:43.115915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df1804-53c4-4c44-9e5e-78cf6f84106b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:43.143458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15927b-aa26-4249-ab37-93229d527ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:43.144123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15927b-aa26-4249-ab37-93229d527ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.144410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15927b-aa26-4249-ab37-93229d527ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.144719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15927b-aa26-4249-ab37-93229d527ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.145849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15927b-aa26-4249-ab37-93229d527ec1 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=3d304d9729a146c6a572afe56c525132) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:43.145849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15927b-aa26-4249-ab37-93229d527ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:43.147304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15927b-aa26-4249-ab37-93229d527ec1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:43.153799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 102/200] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:42 2022] GET /identity/v3/role_assignments?user.id=3d304d9729a146c6a572afe56c525132&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:43.155180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044075e5-017e-4c6c-806d-2bb1ff827af4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.155477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044075e5-017e-4c6c-806d-2bb1ff827af4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.155760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044075e5-017e-4c6c-806d-2bb1ff827af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.174622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-044075e5-017e-4c6c-806d-2bb1ff827af4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:43.175780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-044075e5-017e-4c6c-806d-2bb1ff827af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:43.256014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df1804-53c4-4c44-9e5e-78cf6f84106b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:43.274369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 103/201] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:43.299374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:43.300129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044075e5-017e-4c6c-806d-2bb1ff827af4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:43.307357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:43.308033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.308318 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.308617 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.310187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] RBAC: Adding query filter params (role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=3d304d9729a146c6a572afe56c525132) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:43.310187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:43.311251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:43.314068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:43.319080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 99/202] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:43 2022] GET /identity/v3/role_assignments?user.id=3d304d9729a146c6a572afe56c525132&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:43.428110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044075e5-017e-4c6c-806d-2bb1ff827af4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:43.470395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:43.471317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.471680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.472027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.484213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:43.492309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:43.493448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:43.494434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:43.495999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 104/203] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:43.501868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:43.641587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72151bf1-ba7b-4fa2-bf46-6e12d7e56ac3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:43.688055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:43.688682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.688977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.689260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.690560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:43.690874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:43.692275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:43.701106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 100/204] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:43.707295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:43.791057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:43.834043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5fd1c-0cd6-4735-9567-ac337d4bc0cb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:43.880264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:43.881181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:43.881471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:43.881752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] PATH_INFO: `/v3/users/607819d7e6bf4f4b85abdef3465251d4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:43.885320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] RBAC: Authorizing `identity:get_user(user_id=607819d7e6bf4f4b85abdef3465251d4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:43.886806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:43.888465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 105/205] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:43 2022] GET /identity/v3/users/607819d7e6bf4f4b85abdef3465251d4 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:43.894697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:43.966697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:44.018919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ee595-2d8f-4f4c-a1c3-6c53b7e18895 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:44.066882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:44.067554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:44.067845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:44.068201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:44.147584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:44.149116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:44.217597 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:44.279394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 101/206] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:44.287249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:44.414019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c185b-3451-4698-bf3e-d255bfd27699 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:44.454456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951158c-7934-480a-aaff-2b1c59e76358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:44.455352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951158c-7934-480a-aaff-2b1c59e76358 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:44.455645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951158c-7934-480a-aaff-2b1c59e76358 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:44.455927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951158c-7934-480a-aaff-2b1c59e76358 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:44.457169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951158c-7934-480a-aaff-2b1c59e76358 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:44.458621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951158c-7934-480a-aaff-2b1c59e76358 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:44.460565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 106/207] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:44 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:44.466264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:44.592562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b934-c19c-4c21-a828-aadbfe7f27e3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:44.635736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0435593-6b9a-4909-bba0-13a779f80750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:44.636409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0435593-6b9a-4909-bba0-13a779f80750 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:44.636732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0435593-6b9a-4909-bba0-13a779f80750 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:44.637002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0435593-6b9a-4909-bba0-13a779f80750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:44.637865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0435593-6b9a-4909-bba0-13a779f80750 None admin] RBAC: Adding query filter params (role.id=2e646c3c6fd546cda41230bb92e3df1d, scope.project.id=897b8b7b9c0b4449800d389268bf55b0, user.id=607819d7e6bf4f4b85abdef3465251d4) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:44.638184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0435593-6b9a-4909-bba0-13a779f80750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:44.639758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0435593-6b9a-4909-bba0-13a779f80750 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:44.649800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 102/208] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:44 2022] GET /identity/v3/role_assignments?user.id=607819d7e6bf4f4b85abdef3465251d4&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=2e646c3c6fd546cda41230bb92e3df1d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:45.523230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffecb0-c4bf-498c-8750-e2f79cb83478 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.524204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffecb0-c4bf-498c-8750-e2f79cb83478 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.524737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffecb0-c4bf-498c-8750-e2f79cb83478 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.526963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 107/209] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:45.532773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4487a-6756-4416-b4eb-e24da7e88b1a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.533696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4487a-6756-4416-b4eb-e24da7e88b1a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.534181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4487a-6756-4416-b4eb-e24da7e88b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.552667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ccf4487a-6756-4416-b4eb-e24da7e88b1a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:45.554495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ccf4487a-6756-4416-b4eb-e24da7e88b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:45.580908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:45.598682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7bbb9-21ec-4d06-bb6a-8c737c7f4299 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.599303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7bbb9-21ec-4d06-bb6a-8c737c7f4299 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.599802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7bbb9-21ec-4d06-bb6a-8c737c7f4299 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.601312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 108/210] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:45.606706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d44a-a1ba-4f09-8d2e-5938732a3f35 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.607070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d44a-a1ba-4f09-8d2e-5938732a3f35 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.607433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d44a-a1ba-4f09-8d2e-5938732a3f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.614146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:45.631943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4667d44a-a1ba-4f09-8d2e-5938732a3f35 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:45.633614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4667d44a-a1ba-4f09-8d2e-5938732a3f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:45.640636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 103/211] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:45.677706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4487a-6756-4416-b4eb-e24da7e88b1a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:45.725274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 109/212] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:27:45.733178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7841e-2aa5-4626-a06f-13861d4e51d6 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.733470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7841e-2aa5-4626-a06f-13861d4e51d6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.733752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7841e-2aa5-4626-a06f-13861d4e51d6 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.735375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 110/213] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:45.741113 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b280d-25ef-4005-938f-e78765d8c008 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.741402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b280d-25ef-4005-938f-e78765d8c008 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.741684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b280d-25ef-4005-938f-e78765d8c008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.759483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e72b280d-25ef-4005-938f-e78765d8c008 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:45.760695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e72b280d-25ef-4005-938f-e78765d8c008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:45.803278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4487a-6756-4416-b4eb-e24da7e88b1a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:45.834327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 111/214] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:45.835196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b280d-25ef-4005-938f-e78765d8c008 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:45.847975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:45.849041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.849471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.849893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.861223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:45.869289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:45.870266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:45.871074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 104/215] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:45.873513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:45.888644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:45.902054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:45.938870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b280d-25ef-4005-938f-e78765d8c008 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:45.972475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929786dc-626f-4c3a-beb9-0cde3a9429cf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:45.973009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193dc5-7068-4b5d-8bd8-e5851bdbe9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:45.973822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193dc5-7068-4b5d-8bd8-e5851bdbe9ca None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.974109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193dc5-7068-4b5d-8bd8-e5851bdbe9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.974380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193dc5-7068-4b5d-8bd8-e5851bdbe9ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.984411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b193dc5-7068-4b5d-8bd8-e5851bdbe9ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5b193dc5-7068-4b5d-8bd8-e5851bdbe9ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:45.992797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:45.993661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:45.994503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 112/216] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:27:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:45.995397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3852c6c-371a-4e85-8060-681b66e85e24 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:45.995566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3852c6c-371a-4e85-8060-681b66e85e24 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:45.995821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3852c6c-371a-4e85-8060-681b66e85e24 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:45.997130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 113/217] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:45.999182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3852c6c-371a-4e85-8060-681b66e85e24 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.011143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.012005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.012288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.012569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.024037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:46.032081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:46.033019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 105/218] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:46.038825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.103379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.103628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3852c6c-371a-4e85-8060-681b66e85e24 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.125756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.133273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbffa8f-86cf-47cb-9ccc-d0f6647e6b58 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.137372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedeef8-42d1-4e19-9556-a0e1eee468d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.138040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedeef8-42d1-4e19-9556-a0e1eee468d7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.138326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedeef8-42d1-4e19-9556-a0e1eee468d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.138604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedeef8-42d1-4e19-9556-a0e1eee468d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.139353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedeef8-42d1-4e19-9556-a0e1eee468d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:46.139661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedeef8-42d1-4e19-9556-a0e1eee468d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.141141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedeef8-42d1-4e19-9556-a0e1eee468d7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.149652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 114/219] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.150864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9417d-6d95-49d5-a47c-9bd2c31aa684 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.154416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9417d-6d95-49d5-a47c-9bd2c31aa684 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.154416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9417d-6d95-49d5-a47c-9bd2c31aa684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.167082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.167753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.168040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.168321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.168410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2da9417d-6d95-49d5-a47c-9bd2c31aa684 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:46.169112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:46.169420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.169602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2da9417d-6d95-49d5-a47c-9bd2c31aa684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:46.170817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.179361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 106/220] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.180169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.239774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 115/221] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:46.240652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9417d-6d95-49d5-a47c-9bd2c31aa684 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.287799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474e4c8-f98d-401c-ba99-bd1fec0e5307 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.321850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.322524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.322813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.323095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.323855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:46.324165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.325653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.327027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.334444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 107/222] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.342689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.351988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9417d-6d95-49d5-a47c-9bd2c31aa684 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.367866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.388039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.388936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.389226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.389500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] PATH_INFO: `/v3/users/c41b7767665042c3a988565065c46f0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.403443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] RBAC: Authorizing `identity:get_user(user_id=c41b7767665042c3a988565065c46f0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.405040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.407024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 116/223] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/users/c41b7767665042c3a988565065c46f0d => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.408208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.438648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71220ad-bc11-4bed-917d-efc02e443b77 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.472328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.473233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.473528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.473802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] PATH_INFO: `/v3/users/3d304d9729a146c6a572afe56c525132` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.474988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] RBAC: Authorizing `identity:get_user(user_id=3d304d9729a146c6a572afe56c525132)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.476447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.478117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 108/224] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/users/3d304d9729a146c6a572afe56c525132 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.478117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.509460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b832-8075-44bd-8f60-fd19b4b076e4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.540480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.543792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.544591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.544905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.545185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.546368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.547816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.550110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 117/225] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.550870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.577647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cde34f-61a0-4012-9184-3a30a794edf8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.591620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.612494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.613650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.613941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.614223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] PATH_INFO: `/v3/users/3d304d9729a146c6a572afe56c525132` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.617336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] RBAC: Authorizing `identity:get_user(user_id=3d304d9729a146c6a572afe56c525132)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.618865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.622756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 109/226] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/users/3d304d9729a146c6a572afe56c525132 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.623573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.652576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8e5e-2d7c-4914-817f-4ddc4b6971fc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.687078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.687727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.688013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.688290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.723336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036bad8-b97c-4106-8d4a-b8d6417f7256 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.753045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.755538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.757063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.757725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.758590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.758930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.759202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.760473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.761954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.763666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 110/227] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.766457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.800172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:46.859022 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 118/228] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:46.859864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:46.863995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b97406-170a-42ac-bfee-5363ae44fa1b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.898240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.898845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.899138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.899430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.900149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.901910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.957997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7cd65-2a0b-43bc-8f30-5581f6d03570 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:46.992437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:46.993332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:46.993618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:46.993891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:46.995082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:46.996532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:46.998269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 119/229] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:46 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:46.999068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:47.009245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:47.039502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 111/230] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:27:46 2022] POST /identity/v3/users => generated 297 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:47.039502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:47.103287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:47.111493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96438f-6e72-4a20-89aa-0c8b7942305e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:47.149531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:47.150781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:47.151070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:47.151610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/3d304d9729a146c6a572afe56c525132/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:47.153938 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d23474-313f-4e49-b01b-9caf494da5da None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:47.154089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daadf80bfa8434db5dcc634d31a1449, user_id=3d304d9729a146c6a572afe56c525132, role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:47.155479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:47.190495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:47.511797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:47.511797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:47.511797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:47.511797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:47.511797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:47.511797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 112/231] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:47 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:47.513077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:47.513077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd466b-1d0c-42f9-9bd9-9cca59610d19 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:47.513077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:47.513077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a03cb-3e3d-49ca-9f5c-6656dc8dc105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:47.513408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a03cb-3e3d-49ca-9f5c-6656dc8dc105 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:47.513408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a03cb-3e3d-49ca-9f5c-6656dc8dc105 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:47.513408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a03cb-3e3d-49ca-9f5c-6656dc8dc105 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/3d304d9729a146c6a572afe56c525132/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:47.513408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a03cb-3e3d-49ca-9f5c-6656dc8dc105 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daadf80bfa8434db5dcc634d31a1449, user_id=3d304d9729a146c6a572afe56c525132, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:47.513408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a03cb-3e3d-49ca-9f5c-6656dc8dc105 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:47.515968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 120/232] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:46 2022] PUT /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/3d304d9729a146c6a572afe56c525132/roles/d74c967ed68244238636f7d9bb0517c6 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:47.517041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:47.609955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:47.634111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 113/233] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:47 2022] PUT /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/3d304d9729a146c6a572afe56c525132/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:47.648702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5da60c-df84-4249-ac69-75d95fef6fc6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:47.708789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e8553-a762-41b6-b3b4-d16e064487ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:47.709485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e8553-a762-41b6-b3b4-d16e064487ff None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:47.709776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e8553-a762-41b6-b3b4-d16e064487ff None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:47.710058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e8553-a762-41b6-b3b4-d16e064487ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:47.710932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e8553-a762-41b6-b3b4-d16e064487ff None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=c41b7767665042c3a988565065c46f0d) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:47.711252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e8553-a762-41b6-b3b4-d16e064487ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:47.712759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e8553-a762-41b6-b3b4-d16e064487ff None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:47.719370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 121/234] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:47 2022] GET /identity/v3/role_assignments?user.id=c41b7767665042c3a988565065c46f0d&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:49.438881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2be31-d4d5-46bd-bdb7-58f414fbc96c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:49.440467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2be31-d4d5-46bd-bdb7-58f414fbc96c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:49.441001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2be31-d4d5-46bd-bdb7-58f414fbc96c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:49.442618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 114/235] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:49.447995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15a2fb-89f4-45fb-8c81-1a5f71829c73 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:49.448823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15a2fb-89f4-45fb-8c81-1a5f71829c73 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:49.449319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15a2fb-89f4-45fb-8c81-1a5f71829c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:49.473574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4e15a2fb-89f4-45fb-8c81-1a5f71829c73 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:49.475130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4e15a2fb-89f4-45fb-8c81-1a5f71829c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:49.567237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 122/236] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:49.580941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a52dd1-72ee-4e87-99c9-b320cc3d9b6e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:49.581712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a52dd1-72ee-4e87-99c9-b320cc3d9b6e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:49.584643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a52dd1-72ee-4e87-99c9-b320cc3d9b6e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:49.587300 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 115/237] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:49.592960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8189254-9780-4aa0-9d63-1a808df0b0ad None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:49.593543 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8189254-9780-4aa0-9d63-1a808df0b0ad None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:49.594040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8189254-9780-4aa0-9d63-1a808df0b0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:49.603129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a52dd1-72ee-4e87-99c9-b320cc3d9b6e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:49.629549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e8189254-9780-4aa0-9d63-1a808df0b0ad None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:49.630762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e8189254-9780-4aa0-9d63-1a808df0b0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:49.737633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a52dd1-72ee-4e87-99c9-b320cc3d9b6e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:49.772679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:49.773488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:49.773780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:49.774064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:49.775265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:49.777264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:49.778940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 116/238] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:49.784956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:49.785640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:49.815717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 123/239] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:49.861990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:49.895948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8189254-9780-4aa0-9d63-1a808df0b0ad None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:49.918369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f5fcd-0438-4f6e-a5a2-9168217ca929 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:49.953734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d762e-e95a-4e64-b62b-5b76ddbe6f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:49.954424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d762e-e95a-4e64-b62b-5b76ddbe6f7b None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:49.954713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d762e-e95a-4e64-b62b-5b76ddbe6f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:49.955545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d762e-e95a-4e64-b62b-5b76ddbe6f7b None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:49.956314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d762e-e95a-4e64-b62b-5b76ddbe6f7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:49.964627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d762e-e95a-4e64-b62b-5b76ddbe6f7b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:50.050536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8189254-9780-4aa0-9d63-1a808df0b0ad None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:50.057793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 117/240] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:27:49 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:27:50.058976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1688-fd41-4d05-b7d3-f550c239bfc8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.059273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1688-fd41-4d05-b7d3-f550c239bfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.059559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1688-fd41-4d05-b7d3-f550c239bfc8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.067682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 118/241] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:50.082938 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d3d15-98f2-43b1-b5a9-5d7eb78409ba None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.083258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d3d15-98f2-43b1-b5a9-5d7eb78409ba None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.083518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d3d15-98f2-43b1-b5a9-5d7eb78409ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.100109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:50.100944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.101233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.101512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.114708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:50.119472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f52d3d15-98f2-43b1-b5a9-5d7eb78409ba None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:50.120715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f52d3d15-98f2-43b1-b5a9-5d7eb78409ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:50.122609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:50.123541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:50.124354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:50.125035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 124/242] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:50.129946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:50.218869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:50.260112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 119/243] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:50.261609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01670482-0699-4e5c-ad67-bf03b91b5fd1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.262072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01670482-0699-4e5c-ad67-bf03b91b5fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.262523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01670482-0699-4e5c-ad67-bf03b91b5fd1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.263970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 120/244] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:50.280174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7b863-52bc-4816-b3cf-f03fd8441a2e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.280742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7b863-52bc-4816-b3cf-f03fd8441a2e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.281186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7b863-52bc-4816-b3cf-f03fd8441a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.288372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafcff5-55e8-4347-b574-278b853c3ce8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:50.300364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c5d7b863-52bc-4816-b3cf-f03fd8441a2e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:50.301823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c5d7b863-52bc-4816-b3cf-f03fd8441a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:50.331078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:50.331751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.332048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.332327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.333257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:50.333736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:50.335422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:50.337032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:50.346237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 125/245] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:50.347350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:50.377375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 121/246] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:50.379115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60335f-7722-403c-b213-8f03ebe8af75 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.379708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60335f-7722-403c-b213-8f03ebe8af75 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.380217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60335f-7722-403c-b213-8f03ebe8af75 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.382269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 122/247] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:50.384092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60335f-7722-403c-b213-8f03ebe8af75 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:50.477259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14649a05-e137-4411-bfa7-77e28f33c4f4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:50.511015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60335f-7722-403c-b213-8f03ebe8af75 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:50.570363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:50.596282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451c061-b8aa-4f93-b8e0-8ee9abb39480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:50.597387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c061-b8aa-4f93-b8e0-8ee9abb39480 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.597830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c061-b8aa-4f93-b8e0-8ee9abb39480 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.597969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:50.598312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c061-b8aa-4f93-b8e0-8ee9abb39480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.599605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.600081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.600725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] PATH_INFO: `/v3/users/e6760b99e55f4c338aa75224b063afc6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.610797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451c061-b8aa-4f93-b8e0-8ee9abb39480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-a451c061-b8aa-4f93-b8e0-8ee9abb39480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:50.619638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:50.622484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:27:50.623261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:50.624338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 126/248] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:27:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:50.625965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4434a-9668-4eba-84a0-2181ef0cfb27 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.626440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4434a-9668-4eba-84a0-2181ef0cfb27 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.627535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4434a-9668-4eba-84a0-2181ef0cfb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.672335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-99f4434a-9668-4eba-84a0-2181ef0cfb27 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:50.673408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] RBAC: Authorizing `identity:get_user(user_id=e6760b99e55f4c338aa75224b063afc6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:50.674492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-99f4434a-9668-4eba-84a0-2181ef0cfb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:50.675584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:50.677606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 123/249] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:50 2022] GET /identity/v3/users/e6760b99e55f4c338aa75224b063afc6 => generated 297 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:50.678988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:50.712479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:50.750340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 127/250] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:50.751597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4434a-9668-4eba-84a0-2181ef0cfb27 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:50.790214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a9f3b-92ea-48c5-a592-d67c8d402938 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:50.839264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:50.840698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.840998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.841302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.856176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:50.868026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:50.868977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:50.869783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 124/251] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:27:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:50.870644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:50.871143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4434a-9668-4eba-84a0-2181ef0cfb27 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:50.875039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:50.908500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:50.909516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:50.909978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:50.910440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:50.911426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:50.911917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:50.913495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:50.927553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 128/252] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:27:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:50.928750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:50.979598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:50.996475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a45ec-e74e-44b9-b54d-d32ea0c02f13 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.039085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.039754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.040048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.040497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.059039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee82c5-a782-4f28-ad22-a204ded8b596 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.096912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.098039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.098534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.099122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.110848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:51.119994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:51.121095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:51.121978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:51.123053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.124431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 129/253] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:51.124782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.125523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.161524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:51.230132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:51.243150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831d70-03b5-4f18-b606-0eaeccb4b6ca None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.251826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 125/254] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:51.252757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.281625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.282553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.283602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.284024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.285041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:51.285492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.287052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.296129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 130/255] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:51.297292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.356045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efadb7a-fedc-43af-ba06-b3eb0bd83076 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.390972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.391863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.392151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.392983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] PATH_INFO: `/v3/users/3d304d9729a146c6a572afe56c525132` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.394252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] RBAC: Authorizing `identity:get_user(user_id=3d304d9729a146c6a572afe56c525132)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.395748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.402124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 126/256] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/users/3d304d9729a146c6a572afe56c525132 => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:51.402124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.403379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:51.421734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea400e62-64f8-455b-b674-c2c74bf2c0fd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.458402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.459601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.460136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.460640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.461597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:51.462103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.462587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:51.465075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.475872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 131/257] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:51.477176 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.526513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3b087-287b-4efd-95cf-2351a32ec0ef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.579291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.582494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.582795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.583084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.584909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.587364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.592836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 127/258] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:51.593962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.610305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4e197-e01f-4b67-ae88-c7d6507b6401 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.657911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.659768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.660439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.660905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] PATH_INFO: `/v3/users/3d304d9729a146c6a572afe56c525132` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.662900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] RBAC: Authorizing `identity:get_user(user_id=3d304d9729a146c6a572afe56c525132)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.664836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.667882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 132/259] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/users/3d304d9729a146c6a572afe56c525132 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:51.668846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.693262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:51.709419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2afe0-7582-4093-9d63-9e28d4f327ef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.738726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:51.754120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.755004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.755436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.755859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.785381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66307a-b0d2-44ee-9a63-972913a2b783 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.828200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.829280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.829666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.830586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] PATH_INFO: `/v3/users/c41b7767665042c3a988565065c46f0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.831895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] RBAC: Authorizing `identity:get_user(user_id=c41b7767665042c3a988565065c46f0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.833552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.835510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 133/260] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/users/c41b7767665042c3a988565065c46f0d => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:51.840797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.844195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.846796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:51.934207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:51.955332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becb6cf-45fe-4a70-a54b-871dd201bcd4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:51.985669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 128/261] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:51.986784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:51.993063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:51.993858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:51.994249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:51.994627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:51.995584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] RBAC: Adding query filter params (role.id=2e646c3c6fd546cda41230bb92e3df1d, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=e6760b99e55f4c338aa75224b063afc6) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:51.996001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:51.997673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:52.009070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 134/262] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/role_assignments?user.id=e6760b99e55f4c338aa75224b063afc6&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=2e646c3c6fd546cda41230bb92e3df1d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:52.009985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:52.015640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:52.113603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b748ade-05f5-4469-a883-af5f0cc526b3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:52.156796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:52.157728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.158226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.158652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.242011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:52.244203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:52.269542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9c06-ee5a-4224-ab0f-e1c3b10f8377 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:52.272443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:52.358245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:52.359419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.360547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.362184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.364830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:52.366480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:52.369063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 135/263] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:52 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:52.370117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:52.379149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 129/264] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:52.381162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:52.413954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:52.516518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b635e07-1184-4793-8755-fd3568220e69 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:52.519421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e764dc5-2de9-4393-877b-cbf7721c3b9a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:52.539998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:52.553292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad345e4-3be5-4d40-a983-d73b70a08aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:52.554395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad345e4-3be5-4d40-a983-d73b70a08aa8 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.554688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad345e4-3be5-4d40-a983-d73b70a08aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.555092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad345e4-3be5-4d40-a983-d73b70a08aa8 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/c41b7767665042c3a988565065c46f0d/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.555611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:52.556508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.556835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.557114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad345e4-3be5-4d40-a983-d73b70a08aa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daadf80bfa8434db5dcc634d31a1449, user_id=c41b7767665042c3a988565065c46f0d, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:52.557219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.558310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:52.558612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad345e4-3be5-4d40-a983-d73b70a08aa8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:52.559761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:52.562128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 136/265] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:52 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:52.565506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:52.640302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 130/266] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:52 2022] PUT /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/c41b7767665042c3a988565065c46f0d/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:52.641674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588799da-3366-4b87-8b68-8323dacebf76 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.641971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588799da-3366-4b87-8b68-8323dacebf76 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.642285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588799da-3366-4b87-8b68-8323dacebf76 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.644635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 131/267] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:52.646405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406547-ed39-41c4-ae03-60828d485d6f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.646697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406547-ed39-41c4-ae03-60828d485d6f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.646980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406547-ed39-41c4-ae03-60828d485d6f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.653015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 132/268] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:52.653015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95406547-ed39-41c4-ae03-60828d485d6f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:52.662943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:52.682001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd142f7a-45f2-4bd6-9b2d-4920e9e1a700 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:52.720907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4b18f-e1dc-47d1-ba4e-9b8f60568a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:52.722157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4b18f-e1dc-47d1-ba4e-9b8f60568a82 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.722642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4b18f-e1dc-47d1-ba4e-9b8f60568a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.723148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4b18f-e1dc-47d1-ba4e-9b8f60568a82 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.724705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4b18f-e1dc-47d1-ba4e-9b8f60568a82 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:52.726368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4b18f-e1dc-47d1-ba4e-9b8f60568a82 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:52.728819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 137/269] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:52 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:52.730312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4612e-dc96-4e0e-b3bd-5243d856325b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.730799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4612e-dc96-4e0e-b3bd-5243d856325b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.731273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4612e-dc96-4e0e-b3bd-5243d856325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.761605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fef4612e-dc96-4e0e-b3bd-5243d856325b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:52.763281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fef4612e-dc96-4e0e-b3bd-5243d856325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:52.805641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95406547-ed39-41c4-ae03-60828d485d6f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:52.843462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 138/270] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:52.845246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb17e4-41ba-45ca-a4b1-eb8736223b2d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.846462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb17e4-41ba-45ca-a4b1-eb8736223b2d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.846756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb17e4-41ba-45ca-a4b1-eb8736223b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.848785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:52.849456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:52.849744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:52.850023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:52.850872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] RBAC: Adding query filter params (role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=3d304d9729a146c6a572afe56c525132) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:52.851190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:52.852682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:52.862458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 133/271] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:52 2022] GET /identity/v3/role_assignments?user.id=3d304d9729a146c6a572afe56c525132&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:52.863093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:52.867776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-90cb17e4-41ba-45ca-a4b1-eb8736223b2d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:52.869500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-90cb17e4-41ba-45ca-a4b1-eb8736223b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:52.927104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:52.954389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 139/272] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:52.959630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb17e4-41ba-45ca-a4b1-eb8736223b2d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:52.984530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:52.988278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9c49c-62e1-4f1f-a071-b910e926a311 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:53.078736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:53.078736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:53.079592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:53.080415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:53.081823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=3d304d9729a146c6a572afe56c525132) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:53.081823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:53.089793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:53.097646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 134/273] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:52 2022] GET /identity/v3/role_assignments?user.id=3d304d9729a146c6a572afe56c525132&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:53.099989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:53.112353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb17e4-41ba-45ca-a4b1-eb8736223b2d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:53.160561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:53.161592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:53.161885 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:53.162211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:53.186190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:53.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:53.199146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:53.199911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:53.201090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 140/274] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:53.207722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:53.300974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce90c6-d799-4b3e-a925-fa715fe46cc7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:53.315010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:53.344069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:53.344979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:53.345936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:53.346278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:53.347605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:53.348715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:53.350341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:53.352266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 135/275] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:53.359162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:53.394168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4dfb4-fe71-43a7-b5b9-11cc65e4d2bb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:53.442163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:53.442842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:53.443147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:53.443416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:53.444175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:53.444488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:53.445924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:53.464819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 141/276] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:53.465688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:53.503921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4e34-0705-4273-9040-8be4de74a9eb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:53.544019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee225f0d-da0f-468b-8e4a-497424476e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:53.545553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee225f0d-da0f-468b-8e4a-497424476e36 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:53.546116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee225f0d-da0f-468b-8e4a-497424476e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:53.546590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee225f0d-da0f-468b-8e4a-497424476e36 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:53.547512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee225f0d-da0f-468b-8e4a-497424476e36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:53.549773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee225f0d-da0f-468b-8e4a-497424476e36 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:53.577365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b06a24-4115-4ca1-978d-5d0fdb555ce9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:53.619605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:53.620425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:53.620813 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:53.621097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] PATH_INFO: `/v3/users/b65618ddb07b439b97890e6b6af83772` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:53.624806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:53.635547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] RBAC: Authorizing `identity:get_user(user_id=b65618ddb07b439b97890e6b6af83772)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:53.637058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:53.638996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 142/277] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:53 2022] GET /identity/v3/users/b65618ddb07b439b97890e6b6af83772 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:53.645323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:53.658537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 136/278] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:27:53 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:53.768937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437882fc-0384-43e4-8072-25ac5710624d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:53.808175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e00875-e5ed-4f05-985a-ba59fe04198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:53.809124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e00875-e5ed-4f05-985a-ba59fe04198a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:53.809554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e00875-e5ed-4f05-985a-ba59fe04198a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:53.810009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e00875-e5ed-4f05-985a-ba59fe04198a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:53.810997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e00875-e5ed-4f05-985a-ba59fe04198a None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, user.id=b65618ddb07b439b97890e6b6af83772, scope.system=all) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:53.811459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e00875-e5ed-4f05-985a-ba59fe04198a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:53.813090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e00875-e5ed-4f05-985a-ba59fe04198a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:53.819340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 143/279] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 15 17:27:53 2022] GET /identity/v3/role_assignments?user.id=b65618ddb07b439b97890e6b6af83772&scope.system=all&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:54.250532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72876ad0-32b0-4a5e-8cdd-42d6022559fd None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:54.250833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72876ad0-32b0-4a5e-8cdd-42d6022559fd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:54.251115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72876ad0-32b0-4a5e-8cdd-42d6022559fd None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:54.253229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 137/280] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:54.258634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83a888-29c9-45b1-a6a8-f605960735e5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:54.258931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83a888-29c9-45b1-a6a8-f605960735e5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:54.259221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83a888-29c9-45b1-a6a8-f605960735e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:54.277660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3f83a888-29c9-45b1-a6a8-f605960735e5 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:54.278871 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3f83a888-29c9-45b1-a6a8-f605960735e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:54.352193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:54.371177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 144/281] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:54.407052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72876ad0-32b0-4a5e-8cdd-42d6022559fd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:54.459393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:54.546318 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72876ad0-32b0-4a5e-8cdd-42d6022559fd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:54.594235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:54.595045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:54.595366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:54.595649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:54.610716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:54.619048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:54.619893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:54.620693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 138/282] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:54.629590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83a888-29c9-45b1-a6a8-f605960735e5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:54.802770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83a888-29c9-45b1-a6a8-f605960735e5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:54.857524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:54.858587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:54.859075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:54.859550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:54.860492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:54.861590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:54.863257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:54.875997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 145/283] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:54.881898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:55.041563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:55.048392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2df25-2b00-4e49-8eb0-c1d07d2741ff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:55.101207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdfb9-047f-4827-b3a2-4a848fa5a0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:55.102006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdfb9-047f-4827-b3a2-4a848fa5a0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:55.102293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdfb9-047f-4827-b3a2-4a848fa5a0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:55.102573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdfb9-047f-4827-b3a2-4a848fa5a0b9 None admin] PATH_INFO: `/v3/users/e6760b99e55f4c338aa75224b063afc6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:55.103807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdfb9-047f-4827-b3a2-4a848fa5a0b9 None admin] RBAC: Authorizing `identity:get_user(user_id=e6760b99e55f4c338aa75224b063afc6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:55.105332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdfb9-047f-4827-b3a2-4a848fa5a0b9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:55.107297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 139/284] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:54 2022] GET /identity/v3/users/e6760b99e55f4c338aa75224b063afc6 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:55.113410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:55.197815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9b9dc-985d-4c00-9302-a3e1ac25c54d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:55.198446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9b9dc-985d-4c00-9302-a3e1ac25c54d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:55.198924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9b9dc-985d-4c00-9302-a3e1ac25c54d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:55.200776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 140/285] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:55.205833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c74e37-1914-414d-8407-a4ac13857edb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:55.206375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c74e37-1914-414d-8407-a4ac13857edb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:55.206862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c74e37-1914-414d-8407-a4ac13857edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:55.214049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:55.226908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e1c74e37-1914-414d-8407-a4ac13857edb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:55.229255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e1c74e37-1914-414d-8407-a4ac13857edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:55.297610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa374-99b2-4490-8b05-d52bdbc4ce63 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:55.342682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:55.343662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:55.343948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:55.344229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:55.349309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:55.350791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:55.356234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 146/286] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:55 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:55.366754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:55.374321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 141/287] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:55.443881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c74e37-1914-414d-8407-a4ac13857edb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:55.536728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267d3b8-b052-4da4-a567-0245211bb77a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:55.565326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:55.596523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:55.601309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:55.602228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:55.602523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:55.602808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/e6760b99e55f4c338aa75224b063afc6/roles/2e646c3c6fd546cda41230bb92e3df1d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:55.605545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c74e37-1914-414d-8407-a4ac13857edb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:55.609493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daadf80bfa8434db5dcc634d31a1449, user_id=e6760b99e55f4c338aa75224b063afc6, role_id=2e646c3c6fd546cda41230bb92e3df1d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:55.610975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:55.642460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:55.643263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:55.643548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:55.643827 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:55.654706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:55.665481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:55.666469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:55.667258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 142/288] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:55.672818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:55.700929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 147/289] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:27:55 2022] PUT /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449/users/e6760b99e55f4c338aa75224b063afc6/roles/2e646c3c6fd546cda41230bb92e3df1d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:55.818901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480e8fe-39f4-4a62-adb4-435a93555f0e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:55.874029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:55.874643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:55.874930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:55.875266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:55.876059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:55.876368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:55.877898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:55.893693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 143/290] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:55.899379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.035918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1f2ca-acd4-4ab8-87dd-4559111217d9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:56.075346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f59052-18ce-4029-a74d-10a5360e7406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:56.076279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f59052-18ce-4029-a74d-10a5360e7406 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.076571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f59052-18ce-4029-a74d-10a5360e7406 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.076881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f59052-18ce-4029-a74d-10a5360e7406 None admin] PATH_INFO: `/v3/users/c41b7767665042c3a988565065c46f0d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.079424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f59052-18ce-4029-a74d-10a5360e7406 None admin] RBAC: Authorizing `identity:get_user(user_id=c41b7767665042c3a988565065c46f0d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:56.080961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f59052-18ce-4029-a74d-10a5360e7406 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:56.082902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 148/291] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:55 2022] GET /identity/v3/users/c41b7767665042c3a988565065c46f0d => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:56.088486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.109549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:56.149695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9be47-71be-41cd-adff-635eea08bcad None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.150003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9be47-71be-41cd-adff-635eea08bcad None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.150192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9be47-71be-41cd-adff-635eea08bcad None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.153254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 149/292] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:56.157352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba5bb-6f66-4497-bc7b-fb17d3882df8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.157642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba5bb-6f66-4497-bc7b-fb17d3882df8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.158082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba5bb-6f66-4497-bc7b-fb17d3882df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.175932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4d3ba5bb-6f66-4497-bc7b-fb17d3882df8 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:56.177257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4d3ba5bb-6f66-4497-bc7b-fb17d3882df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:56.213078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b126e-d731-4fd0-9790-b03ee94be08e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:56.255079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6853830-c3c0-4f04-9ccd-245850353c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:56.255749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6853830-c3c0-4f04-9ccd-245850353c6f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.256036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6853830-c3c0-4f04-9ccd-245850353c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.256313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6853830-c3c0-4f04-9ccd-245850353c6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.278431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:56.351896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6853830-c3c0-4f04-9ccd-245850353c6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:56.353421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6853830-c3c0-4f04-9ccd-245850353c6f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:56.355868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 150/293] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:56.368993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:56.390921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba5bb-6f66-4497-bc7b-fb17d3882df8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.463211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 144/294] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:56.466779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1ba61-fd68-44e0-9a30-e56b7c8de3ea None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.467513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1ba61-fd68-44e0-9a30-e56b7c8de3ea None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.468048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1ba61-fd68-44e0-9a30-e56b7c8de3ea None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.470264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 145/295] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:56.471575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f8d5c-c44e-453c-813b-5f7c1e96b2f3 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.471865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f8d5c-c44e-453c-813b-5f7c1e96b2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.472402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f8d5c-c44e-453c-813b-5f7c1e96b2f3 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.476636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 146/296] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:56.476636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f8d5c-c44e-453c-813b-5f7c1e96b2f3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.514732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba5bb-6f66-4497-bc7b-fb17d3882df8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:56.550395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73057c21-3d7b-4bf7-8525-813c62eb2463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:56.551190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73057c21-3d7b-4bf7-8525-813c62eb2463 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.551579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73057c21-3d7b-4bf7-8525-813c62eb2463 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.551854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73057c21-3d7b-4bf7-8525-813c62eb2463 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.562443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73057c21-3d7b-4bf7-8525-813c62eb2463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-73057c21-3d7b-4bf7-8525-813c62eb2463 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:56.570572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:56.571629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:27:56.572587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:27:56.576630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 151/297] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:27:56.578176 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c0f92-f1d9-43b5-a2bb-f5b0f670a710 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.578465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c0f92-f1d9-43b5-a2bb-f5b0f670a710 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.578797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c0f92-f1d9-43b5-a2bb-f5b0f670a710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.595803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2b4c0f92-f1d9-43b5-a2bb-f5b0f670a710 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:56.597059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2b4c0f92-f1d9-43b5-a2bb-f5b0f670a710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:56.604077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f8d5c-c44e-453c-813b-5f7c1e96b2f3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:56.619731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:56.627889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:56.640319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17229-2b52-45c1-bf6a-2f977e488b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:56.641223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17229-2b52-45c1-bf6a-2f977e488b55 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.641603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17229-2b52-45c1-bf6a-2f977e488b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.641889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17229-2b52-45c1-bf6a-2f977e488b55 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.643674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d17229-2b52-45c1-bf6a-2f977e488b55 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:56.645165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d17229-2b52-45c1-bf6a-2f977e488b55 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:56.647036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 147/298] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:56 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:56.648233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc86c09-1c04-4df9-8296-c168a0a8c5a1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.648981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc86c09-1c04-4df9-8296-c168a0a8c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.649249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc86c09-1c04-4df9-8296-c168a0a8c5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.667813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4fc86c09-1c04-4df9-8296-c168a0a8c5a1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:56.669035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4fc86c09-1c04-4df9-8296-c168a0a8c5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:56.673441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 152/299] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:56.674561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c0f92-f1d9-43b5-a2bb-f5b0f670a710 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.762802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 148/300] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:56.763554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc86c09-1c04-4df9-8296-c168a0a8c5a1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.806466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c0f92-f1d9-43b5-a2bb-f5b0f670a710 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:56.842802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:56.843479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.843770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.844048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.844834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:56.845305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:56.847327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:56.861153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 153/301] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:56.861153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.877673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc86c09-1c04-4df9-8296-c168a0a8c5a1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:56.886840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:56.893195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:56.918773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:56.919589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:56.919883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:56.920481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:56.921654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=c41b7767665042c3a988565065c46f0d) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:56.921975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:56.924217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:56.930508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 149/302] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:56 2022] GET /identity/v3/role_assignments?user.id=c41b7767665042c3a988565065c46f0d&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:56.931311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:56.985855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8b8eb-45f3-496c-be69-438f5d9c2a47 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.036331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.037574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.038155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.038645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.040154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:57.042047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:57.048532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834a4b7-a988-48da-a8ff-66606df4e1ee None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.049378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 154/303] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:57.050523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:57.091706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.091706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.091706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.091706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.102221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:57.117444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:57.118462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:57.119266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 150/304] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:57.120444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:57.172941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:57.204809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:57.211169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63940cfe-433a-465d-bccd-e8f1a2a6967e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.255736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322787a-9962-4d67-adc1-b8ac1b6b04de None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.259755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.260666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.260960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.261239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] PATH_INFO: `/v3/users/b65618ddb07b439b97890e6b6af83772` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.262458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] RBAC: Authorizing `identity:get_user(user_id=b65618ddb07b439b97890e6b6af83772)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:57.264312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:57.271553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 155/305] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:57 2022] GET /identity/v3/users/b65618ddb07b439b97890e6b6af83772 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:57.271553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:57.290283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.290950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.291249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.291513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] PATH_INFO: `/v3/groups` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.292212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:57.293712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:57.374275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723b88d-2772-48cf-a575-d7ec2771b33a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.394269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 151/306] 10.222.0.57 () {60 vars in 1134 bytes} [Fri Jul 15 17:27:57 2022] POST /identity/v3/groups => generated 229 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:27:57.406216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:57.423543 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.424243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.424522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.424834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.425603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:57.426823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:57.427452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:57.441732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 156/307] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:57.443077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:57.474803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:57.560810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee70b52-5529-443b-9d92-aa2c4a2760d9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.587062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:57.591626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0840079-f9c1-47c5-a66c-4e2a118b0096 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.606678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.607028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.609250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.609250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] PATH_INFO: `/v3/users/2ddd9080f2924f379e3717494a9f4d51` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.629259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] RBAC: Authorizing `identity:get_user(user_id=2ddd9080f2924f379e3717494a9f4d51)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:57.631038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:57.636791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 157/308] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:57 2022] GET /identity/v3/users/2ddd9080f2924f379e3717494a9f4d51 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:57.638971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:57.639809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93055b7e-95b2-4b24-8e41-07cbe50aad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.640728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055b7e-95b2-4b24-8e41-07cbe50aad65 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.641021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055b7e-95b2-4b24-8e41-07cbe50aad65 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.641306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055b7e-95b2-4b24-8e41-07cbe50aad65 None admin] PATH_INFO: `/v3/system/users/b65618ddb07b439b97890e6b6af83772/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.643748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93055b7e-95b2-4b24-8e41-07cbe50aad65 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b65618ddb07b439b97890e6b6af83772, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:57.645238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93055b7e-95b2-4b24-8e41-07cbe50aad65 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:57.731696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 152/309] 10.222.0.57 () {58 vars in 1334 bytes} [Fri Jul 15 17:27:57 2022] PUT /identity/v3/system/users/b65618ddb07b439b97890e6b6af83772/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:27:57.739152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:57.760965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95098b83-8f3b-4c86-b7ab-d9800b32f2f8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:57.812517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1ca98-2022-46b9-bb42-47dc6ad61c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:57.813579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1ca98-2022-46b9-bb42-47dc6ad61c10 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:57.814637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1ca98-2022-46b9-bb42-47dc6ad61c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:57.815145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1ca98-2022-46b9-bb42-47dc6ad61c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:57.816224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1ca98-2022-46b9-bb42-47dc6ad61c10 None admin] RBAC: Adding query filter params (role.id=2e646c3c6fd546cda41230bb92e3df1d, user.id=2ddd9080f2924f379e3717494a9f4d51, scope.system=all) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:57.816809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1ca98-2022-46b9-bb42-47dc6ad61c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:57.818564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1ca98-2022-46b9-bb42-47dc6ad61c10 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:57.825329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 158/310] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 15 17:27:57 2022] GET /identity/v3/role_assignments?user.id=2ddd9080f2924f379e3717494a9f4d51&scope.system=all&role.id=2e646c3c6fd546cda41230bb92e3df1d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:27:58.116432 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896be950-5aac-4462-88af-e497611e1db3 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:58.116757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896be950-5aac-4462-88af-e497611e1db3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:58.116955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896be950-5aac-4462-88af-e497611e1db3 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:58.118626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 153/311] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:58.123896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1ea0d-352a-438d-9024-e528e0a6c980 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:58.124349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1ea0d-352a-438d-9024-e528e0a6c980 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:58.124544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1ea0d-352a-438d-9024-e528e0a6c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:58.142026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e7d1ea0d-352a-438d-9024-e528e0a6c980 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:58.143221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e7d1ea0d-352a-438d-9024-e528e0a6c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:58.212695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 159/312] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:58.235342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896be950-5aac-4462-88af-e497611e1db3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:58.343316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896be950-5aac-4462-88af-e497611e1db3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:58.384284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:58.387682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:58.388185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:58.388690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:58.400419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:27:58.408907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:27:58.409771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:27:58.410675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:27:58.413512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 154/313] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:27:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:27:58.421307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1ea0d-352a-438d-9024-e528e0a6c980 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:58.505401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:58.526901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1ea0d-352a-438d-9024-e528e0a6c980 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:58.563092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:58.563764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:58.564585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:58.564892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:58.565653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:58.565961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:58.567360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:58.576279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 160/314] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:27:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:58.582534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:58.605718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:58.691592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4cd7-310b-4a15-a8ce-a4e300cb8b41 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:58.735662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:58.736580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:58.736900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:58.737186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] PATH_INFO: `/v3/users/e6760b99e55f4c338aa75224b063afc6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:58.738408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] RBAC: Authorizing `identity:get_user(user_id=e6760b99e55f4c338aa75224b063afc6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:58.739901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:58.741892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 155/315] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:27:58 2022] GET /identity/v3/users/e6760b99e55f4c338aa75224b063afc6 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:58.747106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:58.849536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495105c-6bb1-4347-80a3-11490837efba None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:58.884537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:58.885220 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:58.885520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:58.885805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:58.896849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:58.963468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:58.965298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:59.069821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 161/316] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:27:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:27:59.075863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:59.170461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:59.182403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a59d4-08f4-4236-8c17-046a3839a507 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:59.218142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed34e87-d726-44ef-b323-20a75ff66f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:59.218935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34e87-d726-44ef-b323-20a75ff66f23 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:59.219226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34e87-d726-44ef-b323-20a75ff66f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:59.219509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed34e87-d726-44ef-b323-20a75ff66f23 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:59.220773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed34e87-d726-44ef-b323-20a75ff66f23 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:59.222256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed34e87-d726-44ef-b323-20a75ff66f23 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:59.224280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 156/317] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:27:59 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:59.229996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:59.281063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:59.365600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077c3c2-9f4d-43e2-be65-d7453b657d8e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:59.401826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f0811-ca49-41f8-8551-a314538cbfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:59.402555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f0811-ca49-41f8-8551-a314538cbfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:59.402841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f0811-ca49-41f8-8551-a314538cbfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:59.403111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f0811-ca49-41f8-8551-a314538cbfb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:59.403944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f0811-ca49-41f8-8551-a314538cbfb1 None admin] RBAC: Adding query filter params (role.id=2e646c3c6fd546cda41230bb92e3df1d, scope.project.id=3daadf80bfa8434db5dcc634d31a1449, user.id=e6760b99e55f4c338aa75224b063afc6) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:27:59.404255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f0811-ca49-41f8-8551-a314538cbfb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:59.405710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f0811-ca49-41f8-8551-a314538cbfb1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:59.412238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 162/318] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:27:59 2022] GET /identity/v3/role_assignments?user.id=e6760b99e55f4c338aa75224b063afc6&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=2e646c3c6fd546cda41230bb92e3df1d => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:59.626610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3e76-6fe0-4cb3-9a76-8ec08882a6a5 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:59.627296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3e76-6fe0-4cb3-9a76-8ec08882a6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:59.627778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3e76-6fe0-4cb3-9a76-8ec08882a6a5 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:59.629731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 157/319] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:27:59.641131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee10231-dc8b-4399-84b4-da2b9f33d95f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:59.641754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee10231-dc8b-4399-84b4-da2b9f33d95f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:59.642775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee10231-dc8b-4399-84b4-da2b9f33d95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:59.660779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0ee10231-dc8b-4399-84b4-da2b9f33d95f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:59.661966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0ee10231-dc8b-4399-84b4-da2b9f33d95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:59.732049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 163/320] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:27:59.754093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3e76-6fe0-4cb3-9a76-8ec08882a6a5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:59.858824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3e76-6fe0-4cb3-9a76-8ec08882a6a5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:27:59.872025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf339-d86a-43dd-9386-db586041da3e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:59.872317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf339-d86a-43dd-9386-db586041da3e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:59.872785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf339-d86a-43dd-9386-db586041da3e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:59.874436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 164/321] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:27:59.879694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58f31-34b6-4220-81fe-8fca4af1eb7f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:59.879937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58f31-34b6-4220-81fe-8fca4af1eb7f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:59.880224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58f31-34b6-4220-81fe-8fca4af1eb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:59.897545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:27:59.898349 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:27:59.898637 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:27:59.898938 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:27:59.900083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:27:59.900729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a4f58f31-34b6-4220-81fe-8fca4af1eb7f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:27:59.901859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:27:59.902184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a4f58f31-34b6-4220-81fe-8fca4af1eb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:27:59.908403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 158/322] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:27:59.910796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:27:59.934846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:59.937854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:27:59.979012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 165/323] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:00.002993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f58f31-34b6-4220-81fe-8fca4af1eb7f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.016707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d98947-8d92-4e26-91ba-6f5725a43387 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.051742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc58856-8ee0-49d4-993c-b0fa69e7d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.052409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc58856-8ee0-49d4-993c-b0fa69e7d920 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.052729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc58856-8ee0-49d4-993c-b0fa69e7d920 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.053015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc58856-8ee0-49d4-993c-b0fa69e7d920 None admin] PATH_INFO: `/v3/groups` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.053711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc58856-8ee0-49d4-993c-b0fa69e7d920 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:00.055163 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc58856-8ee0-49d4-993c-b0fa69e7d920 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:00.103799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f58f31-34b6-4220-81fe-8fca4af1eb7f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.137161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.137895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.138188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.138465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.149291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:00.153662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 159/324] 10.222.0.57 () {60 vars in 1134 bytes} [Fri Jul 15 17:27:59 2022] POST /identity/v3/groups => generated 226 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:28:00.154960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d046d0-9c40-4929-814d-57b27bf146c4 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.155256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d046d0-9c40-4929-814d-57b27bf146c4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.155538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d046d0-9c40-4929-814d-57b27bf146c4 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.156882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 160/325] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:00.157250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:00.158096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:00.161636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 166/326] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:00.165003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38119dc-bef0-4a93-9644-68bdd0e9b021 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.165299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38119dc-bef0-4a93-9644-68bdd0e9b021 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.165586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38119dc-bef0-4a93-9644-68bdd0e9b021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.172914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.185894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d38119dc-bef0-4a93-9644-68bdd0e9b021 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:00.187522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d38119dc-bef0-4a93-9644-68bdd0e9b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:00.203630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:00.266430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 161/327] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:00.289043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38119dc-bef0-4a93-9644-68bdd0e9b021 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.299476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad5268-a12a-479d-8001-dda16b3f6286 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.307679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:00.343928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.344909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.345246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.345525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.346277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:00.346589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:00.348044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:00.356900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 167/328] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:00.363268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.415005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38119dc-bef0-4a93-9644-68bdd0e9b021 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.459085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.459896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.460184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.460463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.473510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:00.477026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:00.483545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:00.484495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:28:00.485332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:00.486992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 162/329] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:00.492472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.498694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d5b02-d0dd-4450-8207-c8a0d97130d2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.541432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.542358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.542860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.543276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] PATH_INFO: `/v3/users/b65618ddb07b439b97890e6b6af83772` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.544698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] RBAC: Authorizing `identity:get_user(user_id=b65618ddb07b439b97890e6b6af83772)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:00.546332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:00.549191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 168/330] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:28:00 2022] GET /identity/v3/users/b65618ddb07b439b97890e6b6af83772 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:00.554563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.589179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:00.622888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520fcfd-45a3-45b2-9c01-31f2cdf07eab None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.666872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.667778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.668287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.668772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.669728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:00.670184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:00.671964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:00.674822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f414da-2751-46c1-915f-a6c47dae145f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.681984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 163/331] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:00.691068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.718727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.719633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.720673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.721102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.722097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] RBAC: Adding query filter params (role.id=1d367371231a47c48cb7741c5e4d7ed3, user.id=b65618ddb07b439b97890e6b6af83772, scope.system=all) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:00.722420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:00.723885 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:00.738646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 169/332] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 15 17:28:00 2022] GET /identity/v3/role_assignments?user.id=b65618ddb07b439b97890e6b6af83772&scope.system=all&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:00.815558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d54b-5af0-4be0-b6a5-5568fb5473b2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:00.860637 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a32b-b155-4ea7-82d0-d3e0ecd76b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:00.862481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a32b-b155-4ea7-82d0-d3e0ecd76b70 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:00.862999 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a32b-b155-4ea7-82d0-d3e0ecd76b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:00.863496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a32b-b155-4ea7-82d0-d3e0ecd76b70 None admin] PATH_INFO: `/v3/users/2ddd9080f2924f379e3717494a9f4d51` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:00.865048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a32b-b155-4ea7-82d0-d3e0ecd76b70 None admin] RBAC: Authorizing `identity:get_user(user_id=2ddd9080f2924f379e3717494a9f4d51)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:00.866821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a32b-b155-4ea7-82d0-d3e0ecd76b70 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:00.869089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 164/333] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:28:00 2022] GET /identity/v3/users/2ddd9080f2924f379e3717494a9f4d51 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:00.874411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:00.892337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:01.004396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa727e3-2d0c-49cf-831f-08fc67496fcd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:01.042426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11591ad6-8fa7-4132-b94a-e8d30f7056f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:01.043315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591ad6-8fa7-4132-b94a-e8d30f7056f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:01.043603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591ad6-8fa7-4132-b94a-e8d30f7056f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:01.043882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591ad6-8fa7-4132-b94a-e8d30f7056f9 None admin] PATH_INFO: `/v3/system/users/2ddd9080f2924f379e3717494a9f4d51/roles/2e646c3c6fd546cda41230bb92e3df1d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:01.045447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11591ad6-8fa7-4132-b94a-e8d30f7056f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2ddd9080f2924f379e3717494a9f4d51, role_id=2e646c3c6fd546cda41230bb92e3df1d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:01.046850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11591ad6-8fa7-4132-b94a-e8d30f7056f9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:01.152546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 170/334] 10.222.0.57 () {58 vars in 1334 bytes} [Fri Jul 15 17:28:00 2022] PUT /identity/v3/system/users/2ddd9080f2924f379e3717494a9f4d51/roles/2e646c3c6fd546cda41230bb92e3df1d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:28:02.921410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95597287-c0ad-41a3-ba5e-6da1547990b1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:02.921410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95597287-c0ad-41a3-ba5e-6da1547990b1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:02.921410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95597287-c0ad-41a3-ba5e-6da1547990b1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:02.921410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 165/335] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:02.938342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63976bf0-9bd9-4efa-8996-630a433e4120 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:02.939678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63976bf0-9bd9-4efa-8996-630a433e4120 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:02.940119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63976bf0-9bd9-4efa-8996-630a433e4120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:02.962530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-63976bf0-9bd9-4efa-8996-630a433e4120 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:02.963957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-63976bf0-9bd9-4efa-8996-630a433e4120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:03.046887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 171/336] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:03.077636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95597287-c0ad-41a3-ba5e-6da1547990b1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:03.082469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:03.201761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95597287-c0ad-41a3-ba5e-6da1547990b1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:03.248642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e94a8-1456-4d82-be44-c634d7dab297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:03.249139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e94a8-1456-4d82-be44-c634d7dab297 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:03.249435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e94a8-1456-4d82-be44-c634d7dab297 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:03.249717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e94a8-1456-4d82-be44-c634d7dab297 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:03.266663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e94a8-1456-4d82-be44-c634d7dab297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b21e94a8-1456-4d82-be44-c634d7dab297 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:03.275145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:03.276357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:03.277313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:03.280757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 166/337] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:03.285747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63976bf0-9bd9-4efa-8996-630a433e4120 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:03.296623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:03.395324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97071d7-f04b-42f9-aea4-e8b98f3bc9ed None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:03.395929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97071d7-f04b-42f9-aea4-e8b98f3bc9ed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:03.396418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97071d7-f04b-42f9-aea4-e8b98f3bc9ed None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:03.398018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 167/338] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:28:03.404096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcfd5c-e3f1-4357-a5f3-476c89e72493 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:03.404662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcfd5c-e3f1-4357-a5f3-476c89e72493 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:03.405140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcfd5c-e3f1-4357-a5f3-476c89e72493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:03.441681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-26fcfd5c-e3f1-4357-a5f3-476c89e72493 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:03.443220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26fcfd5c-e3f1-4357-a5f3-476c89e72493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:03.509376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63976bf0-9bd9-4efa-8996-630a433e4120 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:03.588329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:03.597658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:03.598353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:03.598639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:03.598917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:03.599684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:03.599995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:03.601431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:03.610172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 168/339] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:03.618993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 172/340] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:03.625033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcfd5c-e3f1-4357-a5f3-476c89e72493 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:03.651416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:03.759296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcfd5c-e3f1-4357-a5f3-476c89e72493 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:03.820273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb23a3-5dad-4dc7-b46b-ef9fc0ea1a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:03.820840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:03.821809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb23a3-5dad-4dc7-b46b-ef9fc0ea1a44 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:03.822102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb23a3-5dad-4dc7-b46b-ef9fc0ea1a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:03.822389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb23a3-5dad-4dc7-b46b-ef9fc0ea1a44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:03.882631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bbc4d-7b2c-4fc3-85f7-138e5fc3ccee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:03.945987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10914a7-527e-4129-a9af-004dda9f2d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:03.947602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10914a7-527e-4129-a9af-004dda9f2d3f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:03.948531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb23a3-5dad-4dc7-b46b-ef9fc0ea1a44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:03.949337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10914a7-527e-4129-a9af-004dda9f2d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:03.951046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb23a3-5dad-4dc7-b46b-ef9fc0ea1a44 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:03.951533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10914a7-527e-4129-a9af-004dda9f2d3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:03.963574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:03.974334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10914a7-527e-4129-a9af-004dda9f2d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-a10914a7-527e-4129-a9af-004dda9f2d3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:03.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:03.983465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:03.984272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:03.985959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 173/341] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:03.991222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8584-4ab2-4dc0-b280-59144d0cf095 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:03.991512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8584-4ab2-4dc0-b280-59144d0cf095 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:03.991798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8584-4ab2-4dc0-b280-59144d0cf095 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:03.997502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 174/342] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:03.999569 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d8584-4ab2-4dc0-b280-59144d0cf095 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:04.142817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 169/343] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:04.146601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76355f50-b2e2-4831-9ba1-0e08925334d2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.147063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76355f50-b2e2-4831-9ba1-0e08925334d2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.147487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76355f50-b2e2-4831-9ba1-0e08925334d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.175726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-76355f50-b2e2-4831-9ba1-0e08925334d2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:04.176981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-76355f50-b2e2-4831-9ba1-0e08925334d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:04.190527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d8584-4ab2-4dc0-b280-59144d0cf095 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:04.221286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:04.233917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:04.236464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.236464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.236464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.236712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:04.236831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:04.238511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:04.251986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 175/344] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:04.252985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:04.274037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 170/345] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:04.275807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c5a7-1608-4a7a-b328-1fa2bbfb5bc0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.276549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c5a7-1608-4a7a-b328-1fa2bbfb5bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.277049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c5a7-1608-4a7a-b328-1fa2bbfb5bc0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.278732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 171/346] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:04.281794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c5a7-1608-4a7a-b328-1fa2bbfb5bc0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:04.313865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:04.410966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f3cc0-1035-4854-9b9a-afd8171d345d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:04.428539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c5a7-1608-4a7a-b328-1fa2bbfb5bc0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:04.461601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83c2e-d28d-4148-9bb9-73ffc84db3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:04.462747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83c2e-d28d-4148-9bb9-73ffc84db3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.463333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83c2e-d28d-4148-9bb9-73ffc84db3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.464107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83c2e-d28d-4148-9bb9-73ffc84db3b0 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.465543 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83c2e-d28d-4148-9bb9-73ffc84db3b0 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:04.467162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83c2e-d28d-4148-9bb9-73ffc84db3b0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:04.469650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 176/347] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:04 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:04.471416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1683df1-36a7-453b-a896-b0050dc0a57b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.473236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18ca1c-fabb-446e-a87c-51d13f99950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:04.474084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18ca1c-fabb-446e-a87c-51d13f99950f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.474519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18ca1c-fabb-446e-a87c-51d13f99950f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.474960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18ca1c-fabb-446e-a87c-51d13f99950f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.476092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1683df1-36a7-453b-a896-b0050dc0a57b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.476571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1683df1-36a7-453b-a896-b0050dc0a57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.493887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c1683df1-36a7-453b-a896-b0050dc0a57b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:04.495342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c1683df1-36a7-453b-a896-b0050dc0a57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:04.561702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18ca1c-fabb-446e-a87c-51d13f99950f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:04.563564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18ca1c-fabb-446e-a87c-51d13f99950f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:04.567099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:04.594670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 177/348] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:04.596539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1683df1-36a7-453b-a896-b0050dc0a57b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:04.603055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:04.708584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 172/349] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:04.710281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1713-f6a2-42ff-962a-e85eaaafbff3 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.710867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1713-f6a2-42ff-962a-e85eaaafbff3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.710867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1713-f6a2-42ff-962a-e85eaaafbff3 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.712273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 173/350] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:04.712875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d1713-f6a2-42ff-962a-e85eaaafbff3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:04.716944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1683df1-36a7-453b-a896-b0050dc0a57b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:04.760317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:04.761673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.761673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.761673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.776513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:04.782130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:04.783075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:04.783877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 178/351] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:04.784537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:04.836018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d1713-f6a2-42ff-962a-e85eaaafbff3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:04.848700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:04.864560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:04.878060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b55370-1cd2-43a0-a533-70fdbee71856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:04.878933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55370-1cd2-43a0-a533-70fdbee71856 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.879222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55370-1cd2-43a0-a533-70fdbee71856 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.879503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55370-1cd2-43a0-a533-70fdbee71856 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.885658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b55370-1cd2-43a0-a533-70fdbee71856 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:04.887154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b55370-1cd2-43a0-a533-70fdbee71856 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:04.889221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 174/352] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:04 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:04.890483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345e1dc-49c5-4086-893a-1713d0e4c724 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.890780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345e1dc-49c5-4086-893a-1713d0e4c724 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.891064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345e1dc-49c5-4086-893a-1713d0e4c724 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.892343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 175/353] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:04.897769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345e1dc-49c5-4086-893a-1713d0e4c724 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:04.914225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744cfbd-0e56-4f32-abe6-73f5169ee6a6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:04.957086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5d1b7-5f96-4163-8b9d-cc8c0123dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:04.957904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d1b7-5f96-4163-8b9d-cc8c0123dc69 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.958194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d1b7-5f96-4163-8b9d-cc8c0123dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.958473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d1b7-5f96-4163-8b9d-cc8c0123dc69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:04.972704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5d1b7-5f96-4163-8b9d-cc8c0123dc69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6cc5d1b7-5f96-4163-8b9d-cc8c0123dc69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:04.980923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:04.981812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:04.982575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 179/354] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:04.983412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd582877-820b-4f93-82f0-01c47549764a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:04.983663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd582877-820b-4f93-82f0-01c47549764a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:04.984140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd582877-820b-4f93-82f0-01c47549764a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.005709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cd582877-820b-4f93-82f0-01c47549764a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:05.006908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cd582877-820b-4f93-82f0-01c47549764a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:05.024935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345e1dc-49c5-4086-893a-1713d0e4c724 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.066805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.067679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.068067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.068246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.071387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:05.071957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:05.075434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 176/355] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:04 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:05.075434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.090306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 180/356] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:05.092119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31671cd-a17a-44a2-891d-8e8cf500030f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.092417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31671cd-a17a-44a2-891d-8e8cf500030f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.092730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31671cd-a17a-44a2-891d-8e8cf500030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.110178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c31671cd-a17a-44a2-891d-8e8cf500030f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:05.111930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c31671cd-a17a-44a2-891d-8e8cf500030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:05.148175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:05.148454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:05.196558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 181/357] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:05.197528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31671cd-a17a-44a2-891d-8e8cf500030f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.205578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98aa1-9f15-4065-9419-6af31274d60a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.253491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.254183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.254474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.254757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.255681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:05.255957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:05.257620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:05.267096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 177/358] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:05.267749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.332825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31671cd-a17a-44a2-891d-8e8cf500030f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.378127 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.378833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.379085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.379365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.380212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=897b8b7b9c0b4449800d389268bf55b0) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:05.380527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:05.382702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:05.389621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 182/359] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:05.390642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.397501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211337-1d3f-4427-b2ea-05d27904f37b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.424574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:05.431970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:05.452142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.452996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.453357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.453709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.454594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:05.455345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:05.456807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:05.467492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 178/360] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:05.470533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.553874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474ac37-82c9-4119-9b18-b2601c5e77a4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.598478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.599349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.599631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.599909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.601149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:05.602621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:05.604561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 183/361] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:05.609476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.619113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1c23e-00bf-4fac-87bb-22652472f346 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.664002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.664848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.665137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.665415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.676845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:05.688065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:05.689119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 17:28:05.689929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:05.696137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 179/362] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:05.696137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.739578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:05.743079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8b8c8-956b-4b6d-b809-dc048a784b90 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.746891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:05.791150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.791942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.792360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.792571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.805395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:05.815093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:05.816844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 184/363] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:05.817753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:05.828819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535733d-2c8f-469d-ade8-75b8fb2cacc7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.879655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.879655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.879655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.879655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:05.953595 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3fa96-cba9-413e-9e43-9690e4e3c750 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:05.966394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:05.968174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:05.998636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:05.999381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:05.999579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:05.999863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.015532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.033970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.082743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.084173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.086839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 180/364] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:06.088116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.194815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f476-c85d-43e3-83dc-00e0dc80afea None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.204168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 185/365] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:06.204168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.232856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.233510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.233793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.234073 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.234902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=897b8b7b9c0b4449800d389268bf55b0) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:06.235215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.237406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.243421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 181/366] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:06.245802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.250819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.281444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.331940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c24439-1ee5-4594-98fa-fa012b2f96f3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.370904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.371487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.371814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.372034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.372844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:06.373212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.374824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.384518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 186/367] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:06.385325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.400988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792cec7-3fc1-4461-901f-56e2c6e8a6b4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.445235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.445913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.446199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.446478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.447503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:06.447813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.449612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.461986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 182/368] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:06.462957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.502544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b97a32-a5e5-46f9-b93c-457e82a093cb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.539487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.544948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.549069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.550184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.550623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.551647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.553224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.555825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.558489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 187/369] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:06.560023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.584306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7340d8-b661-41c9-943f-64eec90bc598 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.629532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.630823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.631261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.631969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.633698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.635486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.643225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 183/370] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:06.643225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.679609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86beb8bf-8634-4162-acbd-bff16fbbc1aa None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.719586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.721102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.721389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.721817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] PATH_INFO: `/v3/users/2ddd9080f2924f379e3717494a9f4d51` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.723945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] RBAC: Authorizing `identity:get_user(user_id=2ddd9080f2924f379e3717494a9f4d51)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.725438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.732244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 188/371] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/users/2ddd9080f2924f379e3717494a9f4d51 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:06.733106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.758679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a3711-f059-4c91-964d-23298b279f7e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.794114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.795136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.795603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.796089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.799660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.809979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:06.863416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987d8e8-a7e5-4397-93e8-abe1d8bd6db1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:06.868989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.870675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.909946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:06.910744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:06.911034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:06.911311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:06.912524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:06.914530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:06.916983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 189/372] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:06.918462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:06.988704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 184/373] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:06.990543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:07.036771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:07.038832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47304353-14f0-4825-9f28-0e247b8bf7ad None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:07.072983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:07.075873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:07.077019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.077600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.078103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.080117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:07.083225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:07.088928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 190/374] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:07.089975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:07.120075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fbe4c-f9ad-4405-9887-875d64407714 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:07.162306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:07.163275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.163833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.164423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.165630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] RBAC: Adding query filter params (role.id=2e646c3c6fd546cda41230bb92e3df1d, user.id=2ddd9080f2924f379e3717494a9f4d51, scope.system=all) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:07.166170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:07.167996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:07.181885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 185/375] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Jul 15 17:28:06 2022] GET /identity/v3/role_assignments?user.id=2ddd9080f2924f379e3717494a9f4d51&scope.system=all&role.id=2e646c3c6fd546cda41230bb92e3df1d => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:07.183043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:07.230704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4755dff-342f-49eb-a2b6-67b8e4e38419 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:07.295235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:07.296023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.296316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.296614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.297469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=3daadf80bfa8434db5dcc634d31a1449) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:07.297783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:07.299916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:07.315573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 191/376] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:07 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:07.316730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:07.324411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:07.343731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d7a3e4-4605-44f9-8b42-dfc6144a695d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:07.402474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:07.424941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:07.425777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.426069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.426351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] PATH_INFO: `/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.430213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] RBAC: Authorizing `identity:get_project(project_id=6ce7dfbb170044b796f3aa09ded4d2e4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:07.431916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:07.444303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 186/377] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:07 2022] GET /identity/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:07.454037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:07.473534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bdab4-80ac-43f6-b1bf-16775e6b2a58 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:07.519615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9a0bf-1cfb-4d78-a19d-ef408ebcadac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:07.520160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9a0bf-1cfb-4d78-a19d-ef408ebcadac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.520575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9a0bf-1cfb-4d78-a19d-ef408ebcadac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.520897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9a0bf-1cfb-4d78-a19d-ef408ebcadac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.521762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9a0bf-1cfb-4d78-a19d-ef408ebcadac None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=3daadf80bfa8434db5dcc634d31a1449) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:07.522081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9a0bf-1cfb-4d78-a19d-ef408ebcadac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:07.524363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9a0bf-1cfb-4d78-a19d-ef408ebcadac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:07.533976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 192/378] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:07 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:07.534716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f35f8-1c8e-48d7-9cc3-7eaf3921bc59 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.535066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f35f8-1c8e-48d7-9cc3-7eaf3921bc59 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.535350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f35f8-1c8e-48d7-9cc3-7eaf3921bc59 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.540656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 193/379] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:07.546873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4138d8-763a-4b23-b5e2-b25c2244ae68 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.547214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4138d8-763a-4b23-b5e2-b25c2244ae68 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.547449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4138d8-763a-4b23-b5e2-b25c2244ae68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.588647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1d4138d8-763a-4b23-b5e2-b25c2244ae68 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:07.589896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1d4138d8-763a-4b23-b5e2-b25c2244ae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:07.603796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cdd37-69f5-4321-bf4b-60ba434d50b2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:07.641608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:07.643107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.643394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.643674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] PATH_INFO: `/v3/groups/dfbf3bf521f640a18c9fe71e6c9c509e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.651693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] RBAC: Authorizing `identity:get_group(group_id=dfbf3bf521f640a18c9fe71e6c9c509e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:07.653235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:07.655358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 187/380] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:07 2022] GET /identity/v3/groups/dfbf3bf521f640a18c9fe71e6c9c509e => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:07.660968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:07.693076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:07.757926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 194/381] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:07.775144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f5c06-8b5e-4f9e-ad81-8f1c96aeb98d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:07.790449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4138d8-763a-4b23-b5e2-b25c2244ae68 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:07.814800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:07.815473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:07.815761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:07.816042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:07.816913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] RBAC: Adding query filter params (group.id=dfbf3bf521f640a18c9fe71e6c9c509e, role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=6ce7dfbb170044b796f3aa09ded4d2e4) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:07.817231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:07.818704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:07.832887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 188/382] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:07 2022] GET /identity/v3/role_assignments?group.id=dfbf3bf521f640a18c9fe71e6c9c509e&scope.project.id=6ce7dfbb170044b796f3aa09ded4d2e4&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:07.868984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:07.942932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4138d8-763a-4b23-b5e2-b25c2244ae68 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:08.000986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:08.002315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:08.002605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:08.002883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:08.015643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:08.023574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:08.024519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:08.025331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 195/383] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:08.032500 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:08.128764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:08.145180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc29e85-74d3-4b11-8b1f-d661b919c57b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:08.191739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:08.192427 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:08.192736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:08.193012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:08.193766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:08.194070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:08.195649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:08.208975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 189/384] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:08.214609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:08.348354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8058e-a191-4dd8-a048-bf89b613e5fb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:08.364204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:08.385878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:08.386913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:08.388946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:08.388946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:08.388946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:08.390155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:08.392322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 196/385] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:08 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:08.397784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:08.529107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d3ed2-a952-41a0-83cb-df4ca586ad40 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:08.569168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8d9dc-4df9-4a42-afc9-a314d792f8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:08.570422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d9dc-4df9-4a42-afc9-a314d792f8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:08.570907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d9dc-4df9-4a42-afc9-a314d792f8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:08.571380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d9dc-4df9-4a42-afc9-a314d792f8c6 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:08.572801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8d9dc-4df9-4a42-afc9-a314d792f8c6 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:08.574461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8d9dc-4df9-4a42-afc9-a314d792f8c6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:08.576407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 190/386] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:08 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:08.582707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:08.746312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2f192-c9bc-4639-b6eb-1ece7df9fe0f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:08.783483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06299b84-c436-4688-ba99-683e59817cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:08.784369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06299b84-c436-4688-ba99-683e59817cab None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:08.784675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06299b84-c436-4688-ba99-683e59817cab None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:08.784954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06299b84-c436-4688-ba99-683e59817cab None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:08.786905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06299b84-c436-4688-ba99-683e59817cab None admin] RBAC: Authorizing `identity:create_grant(project_id=897b8b7b9c0b4449800d389268bf55b0, group_id=4467db1a7dc248e2a64f3a663a3c9ee6, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:08.788354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06299b84-c436-4688-ba99-683e59817cab None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:08.857108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1c11-8282-4f93-af8c-2a2e12f84295 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:08.857435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1c11-8282-4f93-af8c-2a2e12f84295 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:08.857683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f1c11-8282-4f93-af8c-2a2e12f84295 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:08.859334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 191/387] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:08.864584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221464b-b14d-4682-bfe4-5ef997e88e94 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:08.864912 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221464b-b14d-4682-bfe4-5ef997e88e94 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:08.865200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221464b-b14d-4682-bfe4-5ef997e88e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:08.886355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 197/388] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 15 17:28:08 2022] PUT /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:08.887571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2221464b-b14d-4682-bfe4-5ef997e88e94 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:08.890411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2221464b-b14d-4682-bfe4-5ef997e88e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:08.905575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:08.969853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 192/389] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:08.994178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06299b84-c436-4688-ba99-683e59817cab None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:09.042155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:09.106368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06299b84-c436-4688-ba99-683e59817cab None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:09.144211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:09.145175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:09.145470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:09.145754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:09.157544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:09.165712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:09.166576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:09.167831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 198/390] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:09.173346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221464b-b14d-4682-bfe4-5ef997e88e94 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:09.288147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221464b-b14d-4682-bfe4-5ef997e88e94 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:09.324802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:09.327796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:09.327796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:09.327796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:09.327796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:09.327796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:09.328491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:09.337607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 193/391] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:09.343178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:09.452026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c5da2-ab35-416d-85cf-46269c4fadfb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:09.456700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:09.492739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:09.493603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:09.493888 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:09.494169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:09.495371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:09.497305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:09.503045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 199/392] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:09 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:09.505204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:09.522133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:09.637820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff4810-a6e5-49fd-86ce-be077bacc9bc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:09.698833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78048aa2-47e3-4261-9cce-4e33c4e403db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:09.699705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78048aa2-47e3-4261-9cce-4e33c4e403db None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:09.699980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78048aa2-47e3-4261-9cce-4e33c4e403db None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:09.700350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78048aa2-47e3-4261-9cce-4e33c4e403db None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:09.703454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78048aa2-47e3-4261-9cce-4e33c4e403db None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:09.704966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78048aa2-47e3-4261-9cce-4e33c4e403db None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:09.707031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 194/393] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:09 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:09.712469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:09.875213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568d93-8793-449d-966b-8d6139691893 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:09.915662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb840b1-8189-40af-9575-1f543f669b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:09.916472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb840b1-8189-40af-9575-1f543f669b60 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:09.916824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb840b1-8189-40af-9575-1f543f669b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:09.917104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb840b1-8189-40af-9575-1f543f669b60 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:09.922468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb840b1-8189-40af-9575-1f543f669b60 None admin] RBAC: Authorizing `identity:create_grant(project_id=897b8b7b9c0b4449800d389268bf55b0, group_id=4467db1a7dc248e2a64f3a663a3c9ee6, role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:09.924488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb840b1-8189-40af-9575-1f543f669b60 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:10.014805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 200/394] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 15 17:28:09 2022] PUT /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/d74c967ed68244238636f7d9bb0517c6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:10.081711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a594237-240b-47f9-b5c6-837715789575 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.082127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a594237-240b-47f9-b5c6-837715789575 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.082293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a594237-240b-47f9-b5c6-837715789575 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.084001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 195/395] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:10.099343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805d671-eb57-4b99-b411-f2df39f5736c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.099657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805d671-eb57-4b99-b411-f2df39f5736c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.099916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805d671-eb57-4b99-b411-f2df39f5736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.124303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7805d671-eb57-4b99-b411-f2df39f5736c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:10.125567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7805d671-eb57-4b99-b411-f2df39f5736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:10.151950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8797-7109-4ca1-a4d4-7dfed794d2d9 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.152267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8797-7109-4ca1-a4d4-7dfed794d2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.152528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8797-7109-4ca1-a4d4-7dfed794d2d9 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.154227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 196/396] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:10.156993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:10.160571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df780a-b7e4-479f-8a61-d8a1e0ff6ace None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.160897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df780a-b7e4-479f-8a61-d8a1e0ff6ace None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.161186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df780a-b7e4-479f-8a61-d8a1e0ff6ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.198530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c0df780a-b7e4-479f-8a61-d8a1e0ff6ace None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:10.199688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c0df780a-b7e4-479f-8a61-d8a1e0ff6ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:10.229084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 201/397] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:10.261535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805d671-eb57-4b99-b411-f2df39f5736c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:10.329210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 197/398] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:10.355551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df780a-b7e4-479f-8a61-d8a1e0ff6ace None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:10.366899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:10.448950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805d671-eb57-4b99-b411-f2df39f5736c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:10.498482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:10.499300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.499589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.499867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.506918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df780a-b7e4-479f-8a61-d8a1e0ff6ace None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:10.523862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:10.536192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:10.537225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:10.538020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 202/399] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:10.543875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:10.546532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ae7f5-340d-47cd-b33b-af58b31493c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:10.547358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ae7f5-340d-47cd-b33b-af58b31493c1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.547645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ae7f5-340d-47cd-b33b-af58b31493c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.548478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ae7f5-340d-47cd-b33b-af58b31493c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.577094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ae7f5-340d-47cd-b33b-af58b31493c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-089ae7f5-340d-47cd-b33b-af58b31493c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:10.588095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:10.590301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 198/400] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:10.592118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a0070-9813-469c-8352-27e3cac354fb None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.593662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a0070-9813-469c-8352-27e3cac354fb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.593662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a0070-9813-469c-8352-27e3cac354fb None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.598813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 199/401] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:10.599050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:10.600864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a0070-9813-469c-8352-27e3cac354fb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:10.694048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a1b02-3fbc-4075-ad55-475bee7981d6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:10.713042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:10.730499 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c44c75-df08-47a9-b90a-98a82fcfb943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:10.731038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44c75-df08-47a9-b90a-98a82fcfb943 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.731325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44c75-df08-47a9-b90a-98a82fcfb943 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.733744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c44c75-df08-47a9-b90a-98a82fcfb943 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.733744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44c75-df08-47a9-b90a-98a82fcfb943 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:10.733744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44c75-df08-47a9-b90a-98a82fcfb943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:10.734230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c44c75-df08-47a9-b90a-98a82fcfb943 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:10.742844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a0070-9813-469c-8352-27e3cac354fb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:10.743244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 203/402] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:10.744584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ef5c4-c6ef-4e1f-b2ae-5d7d3ab2a85d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.744908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ef5c4-c6ef-4e1f-b2ae-5d7d3ab2a85d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.745200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ef5c4-c6ef-4e1f-b2ae-5d7d3ab2a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.765326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-483ef5c4-c6ef-4e1f-b2ae-5d7d3ab2a85d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:10.766470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-483ef5c4-c6ef-4e1f-b2ae-5d7d3ab2a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:10.791882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:10.793165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.793916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.794810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.796081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:10.796886 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:10.799405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:10.815587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 200/403] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:10.815587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:10.848726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 204/404] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:10.849632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ef5c4-c6ef-4e1f-b2ae-5d7d3ab2a85d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:10.869892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:10.948849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045821ec-c927-4010-abe2-61ff2ab2d47d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:10.979603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ef5c4-c6ef-4e1f-b2ae-5d7d3ab2a85d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:10.986269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:10.992379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:10.993181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:10.993533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:10.993911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:10.995380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.002612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.002612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 201/405] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:10 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:11.002612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.028331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.030504 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.031237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.031663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.033733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.035612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.043587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 205/406] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:10 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:11.044816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.108551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5304352-3796-41ef-8dfd-354ab68babc3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.148126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.149145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.149533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.149918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.155147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:11.162743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:11.170846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:11.171937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:11.173537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:11.180030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 202/407] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:11.181067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.183190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10425d81-1b36-4726-8ac7-118bd7cb75e8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.231946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.233385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.233841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.234266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.236985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.239246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.241167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 206/408] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:11 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:11.242656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.248785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:11.296697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20a637-4981-4d07-bd52-4ff50a7ae56d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.339386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.340989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.341468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.341928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.344220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.346634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.348817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 203/409] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:11 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:11.349599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.361675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9f999-714a-4461-a66a-7a9b52ca849a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.397829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.398504 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.398790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.399070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.399825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:11.400188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.401911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.416043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 207/410] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:11.416951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.429066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:11.455674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdfdf3-bfa2-4241-beef-31ca37edc7d4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.489671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:11.493121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.494022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.494316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.494600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.496711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] RBAC: Authorizing `identity:create_grant(project_id=3daadf80bfa8434db5dcc634d31a1449, group_id=4467db1a7dc248e2a64f3a663a3c9ee6, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.498367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.523158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0aa8-d2e7-4d64-a515-61fe20c893b8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.556997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047934e-bb9f-45e9-a98b-24425687d925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.557873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047934e-bb9f-45e9-a98b-24425687d925 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.558163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047934e-bb9f-45e9-a98b-24425687d925 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.558445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047934e-bb9f-45e9-a98b-24425687d925 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.560688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047934e-bb9f-45e9-a98b-24425687d925 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daadf80bfa8434db5dcc634d31a1449, group_id=4467db1a7dc248e2a64f3a663a3c9ee6, role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.562119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047934e-bb9f-45e9-a98b-24425687d925 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.597303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 204/411] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 15 17:28:11 2022] PUT /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:11.598140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.658383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 208/412] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 15 17:28:11 2022] PUT /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449/groups/4467db1a7dc248e2a64f3a663a3c9ee6/roles/d74c967ed68244238636f7d9bb0517c6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:11.658383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffbb00-0b09-4570-9412-b3b71519ba7e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.658383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffbb00-0b09-4570-9412-b3b71519ba7e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.658383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffbb00-0b09-4570-9412-b3b71519ba7e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.659267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 209/413] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:11.668561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c0fa4-5d6f-4fa5-acec-25dc0dff9ad1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.670608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c0fa4-5d6f-4fa5-acec-25dc0dff9ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.670911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c0fa4-5d6f-4fa5-acec-25dc0dff9ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.710030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9e9c0fa4-5d6f-4fa5-acec-25dc0dff9ad1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:11.711289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9e9c0fa4-5d6f-4fa5-acec-25dc0dff9ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:11.784563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78ecfa-3baf-400b-9efa-6471a0583cce None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.821461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:11.822339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:11.822631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:11.822917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] PATH_INFO: `/v3/groups/dfbf3bf521f640a18c9fe71e6c9c509e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:11.824388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] RBAC: Authorizing `identity:get_group(group_id=dfbf3bf521f640a18c9fe71e6c9c509e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:11.825933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:11.830085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:11.830655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 205/414] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:11 2022] GET /identity/v3/groups/dfbf3bf521f640a18c9fe71e6c9c509e => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:11.840647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.862275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 210/415] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:11.899210 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c0fa4-5d6f-4fa5-acec-25dc0dff9ad1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:11.903067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:11.962668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccd53e-3277-4804-a198-0d7f871278ba None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:11.999408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:12.000396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.000781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.001663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] PATH_INFO: `/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.006365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] RBAC: Authorizing `identity:get_project(project_id=6ce7dfbb170044b796f3aa09ded4d2e4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:12.009403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:12.016208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 206/416] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:11 2022] GET /identity/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:12.017829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:12.022367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c0fa4-5d6f-4fa5-acec-25dc0dff9ad1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:12.070285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:12.071271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.071651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.072084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.086554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:12.098342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:12.099281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:12.100490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 211/417] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:12.106689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:12.163259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d597a-f6d2-4652-b554-1589b0bbb0e2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:12.199062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:12.207370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:12.232060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a30ea-4a3a-40bc-a088-dcd6f8687303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:12.232997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a30ea-4a3a-40bc-a088-dcd6f8687303 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.233270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a30ea-4a3a-40bc-a088-dcd6f8687303 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.233564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a30ea-4a3a-40bc-a088-dcd6f8687303 None admin] PATH_INFO: `/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4/groups/dfbf3bf521f640a18c9fe71e6c9c509e/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.245179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a081e1-754d-4652-b8c8-1f5e5517279b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:12.245589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a30ea-4a3a-40bc-a088-dcd6f8687303 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ce7dfbb170044b796f3aa09ded4d2e4, group_id=dfbf3bf521f640a18c9fe71e6c9c509e, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:12.249998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a30ea-4a3a-40bc-a088-dcd6f8687303 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:12.284378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447f206-6a4d-4a38-9ace-4ae994de264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:12.285086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447f206-6a4d-4a38-9ace-4ae994de264b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.285380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447f206-6a4d-4a38-9ace-4ae994de264b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.285663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447f206-6a4d-4a38-9ace-4ae994de264b None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.288911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447f206-6a4d-4a38-9ace-4ae994de264b None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:12.288911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447f206-6a4d-4a38-9ace-4ae994de264b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:12.288911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447f206-6a4d-4a38-9ace-4ae994de264b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:12.305835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 212/418] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:12.305835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e954498-d2c4-4a3d-b543-e7ddd5ecb74b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.305835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e954498-d2c4-4a3d-b543-e7ddd5ecb74b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.305835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e954498-d2c4-4a3d-b543-e7ddd5ecb74b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.307084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 213/419] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:12.307629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e954498-d2c4-4a3d-b543-e7ddd5ecb74b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:12.369688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 207/420] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Jul 15 17:28:12 2022] PUT /identity/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4/groups/dfbf3bf521f640a18c9fe71e6c9c509e/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:28:12.371492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629199c-b417-44b7-ae2e-3086d8372d19 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.371989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629199c-b417-44b7-ae2e-3086d8372d19 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.372663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629199c-b417-44b7-ae2e-3086d8372d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.421745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0629199c-b417-44b7-ae2e-3086d8372d19 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:12.423842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0629199c-b417-44b7-ae2e-3086d8372d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:12.450435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e954498-d2c4-4a3d-b543-e7ddd5ecb74b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:12.475250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:12.492584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:12.493465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.493758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.494075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.580136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:12.581682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:12.586334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 208/421] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:12.617223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629199c-b417-44b7-ae2e-3086d8372d19 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:12.709203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 214/422] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:12.714772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:12.750707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:12.782244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629199c-b417-44b7-ae2e-3086d8372d19 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:12.804408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:12.844903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:12.845724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.846007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.846283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.864995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:12.873653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:12.874556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:12.875485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 209/423] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:12.882713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:12.902232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90303-57ef-4a98-862d-73b1e6b639a4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:12.981384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:12.982174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:12.982457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:12.982731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:12.986398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:12.990051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:12.992052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 215/424] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:12 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:12.998136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:13.074017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb8363-e81d-4766-8ba7-ede0631fea38 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:13.140999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:13.149522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:13.150213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:13.153657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:13.153657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:13.153657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:13.153657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:13.153657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:13.160111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ec201-718e-41ec-87f7-3f87a2fe98bd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:13.174664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 210/425] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:13.179820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:13.234421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:13.236415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:13.236736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:13.237021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:13.238999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:13.246619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:13.250244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 216/426] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:12 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:13.252550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:13.255820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:13.383366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c2306-1d63-4532-8cd2-cb826114744b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:13.433328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:13.434373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:13.434862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:13.435355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:13.451557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354422b-8b8e-4784-8c49-230a5785fd5f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:13.508903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:13.510724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:13.525861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:13.526798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:13.527088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:13.527362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:13.528512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=897b8b7b9c0b4449800d389268bf55b0) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:13.528512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:13.529989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:13.548634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 217/427] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:13 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 611 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:13.553853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:13.655322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 211/428] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:28:13.665780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:13.709951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:13.823576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247790f-31c5-4ab6-ba17-f6ecfa5c862b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:13.865145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48dc8a-d2c9-4d78-a94d-87c358b8d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:13.866021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48dc8a-d2c9-4d78-a94d-87c358b8d999 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:13.866302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48dc8a-d2c9-4d78-a94d-87c358b8d999 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:13.866577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48dc8a-d2c9-4d78-a94d-87c358b8d999 None admin] PATH_INFO: `/v3/projects/897b8b7b9c0b4449800d389268bf55b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:13.868201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48dc8a-d2c9-4d78-a94d-87c358b8d999 None admin] RBAC: Authorizing `identity:get_project(project_id=897b8b7b9c0b4449800d389268bf55b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:13.869688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48dc8a-d2c9-4d78-a94d-87c358b8d999 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:13.871403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 218/429] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:13 2022] GET /identity/v3/projects/897b8b7b9c0b4449800d389268bf55b0 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:13.877485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:13.961223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bceda-07ab-4018-ae3c-93a584101b32 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:13.961537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bceda-07ab-4018-ae3c-93a584101b32 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:13.961785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bceda-07ab-4018-ae3c-93a584101b32 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:13.963417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 219/430] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:13.968449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65c73-077c-4c38-ae7a-7d80d638c6cb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:13.968769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65c73-077c-4c38-ae7a-7d80d638c6cb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:13.969060 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65c73-077c-4c38-ae7a-7d80d638c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:13.998938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a3c65c73-077c-4c38-ae7a-7d80d638c6cb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:14.001700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a3c65c73-077c-4c38-ae7a-7d80d638c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:14.009464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901a6fd-831e-47d4-a4fc-882e8615636c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.047568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.049308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.049600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.049885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.051640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:14.053425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:14.055395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 212/431] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:13 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:14.060794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:14.064046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:14.108623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 220/432] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:14.109881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b4397-c7e8-4bc6-b1e4-25ad1f77c76b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.110166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b4397-c7e8-4bc6-b1e4-25ad1f77c76b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.110497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b4397-c7e8-4bc6-b1e4-25ad1f77c76b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.111984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 221/433] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:14.117316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670481ef-710f-4494-8ca8-72a3021c7643 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.117603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670481ef-710f-4494-8ca8-72a3021c7643 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.117879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670481ef-710f-4494-8ca8-72a3021c7643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.134965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-670481ef-710f-4494-8ca8-72a3021c7643 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:14.136797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-670481ef-710f-4494-8ca8-72a3021c7643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:14.152254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:14.201343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955be8d5-59df-45f2-a00a-9ebe7a08e1ab None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.242037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 222/434] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:14.243122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670481ef-710f-4494-8ca8-72a3021c7643 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:14.246055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.246730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.247019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.247299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.248143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=897b8b7b9c0b4449800d389268bf55b0) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:14.248459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:14.249963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:14.256831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 213/435] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:14 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=897b8b7b9c0b4449800d389268bf55b0&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:14.281575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:14.371626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:14.387686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670481ef-710f-4494-8ca8-72a3021c7643 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.412930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4101a17-4363-4294-9237-870a0e31fd5e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.430928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.432260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.432791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.433261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.446425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:14.449799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.450614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.450898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.451175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:14.456536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:14.457440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 17:28:14.458211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:14.461068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 223/436] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:14.462892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:14.467264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:14.471769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:14.472815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:14.473676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 214/437] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:14.479114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:14.482572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:14.603435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d37e8-031a-4949-b706-5f1d72adf6dc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.608466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ac72-2151-4066-a916-0aa1ebcc743c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.649489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:14.661583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.667702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.667702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.667702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.667702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] RBAC: Adding query filter params (name=member) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:14.668448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.668546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:14.668654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.668654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.668654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.668875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:14.668948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:14.668948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:14.668948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:14.678134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 215/438] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:14.678910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 224/439] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:28:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:14.683597 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:14.685448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:14.783542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:14.803448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c572954-8f2f-4b4b-9924-42f92449caef None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.822444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9b5ab-d5bb-49c4-9aa8-f939ef06c365 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:14.839096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c86ac1-5ec1-411f-beba-a837df7aca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.839742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c86ac1-5ec1-411f-beba-a837df7aca63 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.839958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c86ac1-5ec1-411f-beba-a837df7aca63 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.840244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c86ac1-5ec1-411f-beba-a837df7aca63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.868096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dd70e-73bc-4903-b3d6-54b4761261e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:14.869458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd70e-73bc-4903-b3d6-54b4761261e6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:14.870228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd70e-73bc-4903-b3d6-54b4761261e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:14.870703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dd70e-73bc-4903-b3d6-54b4761261e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:14.897808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:14.911661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c86ac1-5ec1-411f-beba-a837df7aca63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:14.912251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c86ac1-5ec1-411f-beba-a837df7aca63 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:14.950336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45dd70e-73bc-4903-b3d6-54b4761261e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:14.951786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45dd70e-73bc-4903-b3d6-54b4761261e6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.015028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 216/440] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:28:15.016360 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e08c5e-c098-481e-9a51-41c821238d51 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.017251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e08c5e-c098-481e-9a51-41c821238d51 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.017538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e08c5e-c098-481e-9a51-41c821238d51 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.018820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 217/441] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:15.021095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e08c5e-c098-481e-9a51-41c821238d51 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.028343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.059335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 225/442] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:28:15.060807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e7d4d-0df8-4360-9652-182a5abfa7a0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.061471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e7d4d-0df8-4360-9652-182a5abfa7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.061751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e7d4d-0df8-4360-9652-182a5abfa7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.078048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-be2e7d4d-0df8-4360-9652-182a5abfa7a0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:15.079803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-be2e7d4d-0df8-4360-9652-182a5abfa7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:15.111694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.124188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e08c5e-c098-481e-9a51-41c821238d51 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.147649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 226/443] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:15.148491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e7d4d-0df8-4360-9652-182a5abfa7a0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.176106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.176927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.177220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.177502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.178836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.180309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.182041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 218/444] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:15.182803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.249125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.252091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e7d4d-0df8-4360-9652-182a5abfa7a0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.283765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d2307-8b3b-4759-b876-3377f23584d3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.287074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.287924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.288204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.289049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] PATH_INFO: `/v3/projects/3daadf80bfa8434db5dcc634d31a1449` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.289695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] RBAC: Authorizing `identity:get_project(project_id=3daadf80bfa8434db5dcc634d31a1449)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.291126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.293079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 227/445] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/projects/3daadf80bfa8434db5dcc634d31a1449 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:15.293600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.320403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.321224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.321507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.321782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.333049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:15.340729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:15.341629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:15.342421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 219/446] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:15.343065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.361355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.401374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d84822-e038-458b-9991-9168e7fc9548 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.435288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.436066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.436347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.436641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.437812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.439379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.440960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20521480-9f09-4ae2-a234-9e7fcae8df96 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.441105 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 228/447] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:15.441857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.475127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.475986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.476276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.476556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] PATH_INFO: `/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.477844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.477976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] RBAC: Authorizing `identity:get_group(group_id=4467db1a7dc248e2a64f3a663a3c9ee6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.479213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.480910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 220/448] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/groups/4467db1a7dc248e2a64f3a663a3c9ee6 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:15.481669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.544806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6caa-2c5a-4fa5-88a1-905d90654b60 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.564641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.579434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.580118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.580408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.580495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae1db3-6b32-4b7f-a169-64467cc14ec3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.580718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.581473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:15.581786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.583156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.591372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 229/449] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:15.592168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.615174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.615836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.616126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.616409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.617283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=3daadf80bfa8434db5dcc634d31a1449) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:15.617609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.619061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.624829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 221/450] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:15.625610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:15.701542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.706519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eb190-e0f9-42ae-aaae-89522ccd7fd8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.742643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561184cc-e62a-4a55-a2e6-e2bcf7245118 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:15.744830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.745800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.746092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.746376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.747241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] RBAC: Adding query filter params (group.id=4467db1a7dc248e2a64f3a663a3c9ee6, role.id=1d367371231a47c48cb7741c5e4d7ed3, scope.project.id=3daadf80bfa8434db5dcc634d31a1449) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:15.747558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.749086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.756873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 230/451] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/role_assignments?group.id=4467db1a7dc248e2a64f3a663a3c9ee6&scope.project.id=3daadf80bfa8434db5dcc634d31a1449&role.id=1d367371231a47c48cb7741c5e4d7ed3 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:15.784231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:15.784937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:15.785227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:15.785511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:15.806869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:15.870851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:15.872353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:15.983674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 222/452] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:15.989495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:16.097324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f503e-5115-41d4-a15e-29ea4ec1c9df None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:16.132248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:16.133155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:16.133451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:16.133739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] PATH_INFO: `/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:16.134935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] RBAC: Authorizing `identity:get_project(project_id=6ce7dfbb170044b796f3aa09ded4d2e4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:16.136402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:16.138311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 231/453] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:28:15 2022] GET /identity/v3/projects/6ce7dfbb170044b796f3aa09ded4d2e4 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:16.144656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:16.195347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:16.246564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a4e09-67f9-4873-9ff9-a158070b9f89 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:16.280110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb422-6f2a-4516-85f7-4dd4b53f50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:16.280990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb422-6f2a-4516-85f7-4dd4b53f50c0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:16.281279 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb422-6f2a-4516-85f7-4dd4b53f50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:16.281560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb422-6f2a-4516-85f7-4dd4b53f50c0 None admin] PATH_INFO: `/v3/groups/dfbf3bf521f640a18c9fe71e6c9c509e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:16.282742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb422-6f2a-4516-85f7-4dd4b53f50c0 None admin] RBAC: Authorizing `identity:get_group(group_id=dfbf3bf521f640a18c9fe71e6c9c509e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:16.284188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb422-6f2a-4516-85f7-4dd4b53f50c0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:16.285839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 223/454] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Jul 15 17:28:16 2022] GET /identity/v3/groups/dfbf3bf521f640a18c9fe71e6c9c509e => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:16.291699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:16.323560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:16.389889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041b1d-96d3-491a-bf4f-b7e40705e7c7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:16.422006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08eee17-be90-4aed-a0fd-6d27fbe427c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:16.423159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08eee17-be90-4aed-a0fd-6d27fbe427c0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:16.423447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08eee17-be90-4aed-a0fd-6d27fbe427c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:16.423724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08eee17-be90-4aed-a0fd-6d27fbe427c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:16.424550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08eee17-be90-4aed-a0fd-6d27fbe427c0 None admin] RBAC: Adding query filter params (group.id=dfbf3bf521f640a18c9fe71e6c9c509e, role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=6ce7dfbb170044b796f3aa09ded4d2e4) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:16.424890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08eee17-be90-4aed-a0fd-6d27fbe427c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:16.426327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08eee17-be90-4aed-a0fd-6d27fbe427c0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:16.431991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 232/455] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Jul 15 17:28:16 2022] GET /identity/v3/role_assignments?group.id=dfbf3bf521f640a18c9fe71e6c9c509e&scope.project.id=6ce7dfbb170044b796f3aa09ded4d2e4&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:18.803579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc950a-68c8-422b-8083-62f539cc0375 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:18.804495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc950a-68c8-422b-8083-62f539cc0375 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:18.804980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc950a-68c8-422b-8083-62f539cc0375 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:18.806780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 224/456] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:18.812525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadb9e7-b126-4ecc-b3d6-41fdc2e37a45 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:18.813923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadb9e7-b126-4ecc-b3d6-41fdc2e37a45 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:18.814441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dadb9e7-b126-4ecc-b3d6-41fdc2e37a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:18.836273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8dadb9e7-b126-4ecc-b3d6-41fdc2e37a45 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:18.837523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8dadb9e7-b126-4ecc-b3d6-41fdc2e37a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:18.916375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 233/457] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:18.962065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc950a-68c8-422b-8083-62f539cc0375 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:18.977458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d3959-884d-4d4a-a483-9a65c98cb676 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:18.978045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d3959-884d-4d4a-a483-9a65c98cb676 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:18.978521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d3959-884d-4d4a-a483-9a65c98cb676 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:18.980380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 234/458] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:28:18.985580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88de8b-202a-49d1-bd41-559f31da113c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:18.986115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88de8b-202a-49d1-bd41-559f31da113c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:18.986594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88de8b-202a-49d1-bd41-559f31da113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.015205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6a88de8b-202a-49d1-bd41-559f31da113c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:19.018048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6a88de8b-202a-49d1-bd41-559f31da113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:19.057043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:19.143776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc950a-68c8-422b-8083-62f539cc0375 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:19.146937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 235/459] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:19.171115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88de8b-202a-49d1-bd41-559f31da113c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:19.182429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:19.221538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907a89f-875f-4338-8246-859b48300725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:19.222354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907a89f-875f-4338-8246-859b48300725 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.222645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907a89f-875f-4338-8246-859b48300725 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.222926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907a89f-875f-4338-8246-859b48300725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.252900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907a89f-875f-4338-8246-859b48300725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:19.254433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907a89f-875f-4338-8246-859b48300725 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6907a89f-875f-4338-8246-859b48300725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:19.296930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:19.298444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:19.299310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:19.299310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:19.299310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:19.299310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:19.299310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 225/460] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:19.304982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907a89f-875f-4338-8246-859b48300725 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:19.319716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88de8b-202a-49d1-bd41-559f31da113c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:19.354313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124a3bd-46ea-49ba-84e8-3b0a18e385ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:19.355271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124a3bd-46ea-49ba-84e8-3b0a18e385ef None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.355614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124a3bd-46ea-49ba-84e8-3b0a18e385ef None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.355897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124a3bd-46ea-49ba-84e8-3b0a18e385ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.367534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124a3bd-46ea-49ba-84e8-3b0a18e385ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:19.369268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124a3bd-46ea-49ba-84e8-3b0a18e385ef None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f124a3bd-46ea-49ba-84e8-3b0a18e385ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:19.389312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:19.397402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 236/461] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb9298-77c9-4d67-af5b-876711982c3b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb9298-77c9-4d67-af5b-876711982c3b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb9298-77c9-4d67-af5b-876711982c3b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.398424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 237/462] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:19.399101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb9298-77c9-4d67-af5b-876711982c3b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:19.408718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:19.465447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907a89f-875f-4338-8246-859b48300725 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:19.504438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7fa13-9457-4d4b-9aa8-ba37e5d2981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:19.505314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7fa13-9457-4d4b-9aa8-ba37e5d2981d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.505673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7fa13-9457-4d4b-9aa8-ba37e5d2981d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.506025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7fa13-9457-4d4b-9aa8-ba37e5d2981d None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.506849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7fa13-9457-4d4b-9aa8-ba37e5d2981d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:19.507233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7fa13-9457-4d4b-9aa8-ba37e5d2981d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:19.508720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7fa13-9457-4d4b-9aa8-ba37e5d2981d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:19.523462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 226/463] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:19.525321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8baf-140b-4f54-a55a-94814cced500 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.526054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8baf-140b-4f54-a55a-94814cced500 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.526889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8baf-140b-4f54-a55a-94814cced500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.541956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb9298-77c9-4d67-af5b-876711982c3b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:19.547112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-01aa8baf-140b-4f54-a55a-94814cced500 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:19.549036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-01aa8baf-140b-4f54-a55a-94814cced500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:19.573746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:19.588455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:19.589421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.589882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.590362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.591272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:19.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:19.593814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:19.617071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 238/464] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:19.618455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:19.626567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 227/465] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:19.632632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa8baf-140b-4f54-a55a-94814cced500 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:19.709324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:19.746886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7062-2d96-4f19-9c2b-27d478f10dc8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:19.772933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa8baf-140b-4f54-a55a-94814cced500 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:19.782997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:19.783696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.783978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.784474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.784993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:19.786440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:19.809725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc6a7c-cd73-4d86-9e85-b91e9d266a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:19.810487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc6a7c-cd73-4d86-9e85-b91e9d266a1d None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.810843 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc6a7c-cd73-4d86-9e85-b91e9d266a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.811189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc6a7c-cd73-4d86-9e85-b91e9d266a1d None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.811965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc6a7c-cd73-4d86-9e85-b91e9d266a1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:19.813605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc6a7c-cd73-4d86-9e85-b91e9d266a1d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:19.879731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 239/466] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:28:19 2022] POST /identity/v3/users => generated 246 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:19.880491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:19.907623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 228/467] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:28:19 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:19.908842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47ac2-fd1d-4254-98da-575c3f35cf03 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.909127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47ac2-fd1d-4254-98da-575c3f35cf03 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.909413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47ac2-fd1d-4254-98da-575c3f35cf03 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.911872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 229/468] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:19.917176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6766f2-6ad6-418a-8973-07c4324f1b0a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:19.917962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6766f2-6ad6-418a-8973-07c4324f1b0a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:19.918346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6766f2-6ad6-418a-8973-07c4324f1b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:19.939453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5c6766f2-6ad6-418a-8973-07c4324f1b0a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:19.941157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5c6766f2-6ad6-418a-8973-07c4324f1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:19.952984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:20.029140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 230/469] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:20.052450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6766f2-6ad6-418a-8973-07c4324f1b0a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:20.103225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2f32f-dd5a-49c1-98ff-cdabef851ea3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:20.149401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:20.171131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e04d6-3b50-4e97-a7fb-8ebde6aa8b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:20.171674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e04d6-3b50-4e97-a7fb-8ebde6aa8b37 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.172724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e04d6-3b50-4e97-a7fb-8ebde6aa8b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.173148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e04d6-3b50-4e97-a7fb-8ebde6aa8b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.191746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e04d6-3b50-4e97-a7fb-8ebde6aa8b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:20.194283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e04d6-3b50-4e97-a7fb-8ebde6aa8b37 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:20.202980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6766f2-6ad6-418a-8973-07c4324f1b0a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-094e04d6-3b50-4e97-a7fb-8ebde6aa8b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:20.217036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:20.217918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:20.218771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:20.218771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:20.218771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:20.218771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:20.218771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 240/470] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:20.220017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e41af2-0605-4223-9099-425154470b8a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.220134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e41af2-0605-4223-9099-425154470b8a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.220443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e41af2-0605-4223-9099-425154470b8a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.221969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 241/471] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:20.231540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e19ef7-ad77-4b6d-9ead-cd311962d497 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.231833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e19ef7-ad77-4b6d-9ead-cd311962d497 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.232115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e19ef7-ad77-4b6d-9ead-cd311962d497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.253393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-00e19ef7-ad77-4b6d-9ead-cd311962d497 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:20.254756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-00e19ef7-ad77-4b6d-9ead-cd311962d497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:20.258512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:20.259515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.259958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.260379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.272419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:20.274137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:20.277408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:20.295501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:20.296553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:20.297691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:20.297691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:20.297691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:20.297691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:20.299595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 231/472] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:20.300638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:20.336898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 242/473] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:20.338843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e19ef7-ad77-4b6d-9ead-cd311962d497 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:20.425320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34343970-227d-45fc-9700-ed48adf61895 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:20.461555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:20.462366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.462754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.463136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.463968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:20.464383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:20.465921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:20.470150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e19ef7-ad77-4b6d-9ead-cd311962d497 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:20.482723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 232/474] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:20.483304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:20.484099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:20.509739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:20.510548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.510892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.511183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.511924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:20.512240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:20.513937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:20.527581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 243/475] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:20.529098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:20.543198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:20.597545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930cf73-f327-47a8-983f-330503f265fc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:20.637291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:20.637979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.638271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.638554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.639315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:20.640759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:20.647463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881fec1-5103-413a-9824-3d1c3e82397e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:20.676682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.sql.core [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 17:28:20.676682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 17:28:20.676682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [parameters: {'id': '9e710fe34794417a9812ee57eae21873', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 17:28:20.676682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80472) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 17:28:20.688780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:20.689721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.690269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.690752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.691748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 15 17:28:20.692757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return ref.to_dict() Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 17:28:20.693574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application yield resource Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise self.value Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application session.commit() Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 17:28:20.694487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.flush() Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application compat.raise_( Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 17:28:20.695331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 17:28:20.696240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 17:28:20.701720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [parameters: {'id': '9e710fe34794417a9812ee57eae21873', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:20.703565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 17:28:20.707731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:20.710975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 233/476] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 15 17:28:20 2022] POST /identity/v3/roles => generated 144 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:20.710975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:20.711374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:20.816690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961840-2ca8-48e6-9f48-2f4f00c7dc6e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:20.817458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 244/477] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:28:20 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:20.819407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:20.859492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:20.872533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:20.889836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:20.890821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:20.891106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:20.891932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:20.893655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:20.895147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:20.982108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a8113-452d-4e07-a174-c10f0aa25dde None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:21.001712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 234/478] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:28:20 2022] POST /identity/v3/users => generated 249 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:21.038653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02501599-af78-488c-9fa0-9a5585f4f3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:21.039847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02501599-af78-488c-9fa0-9a5585f4f3ea None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:21.040285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02501599-af78-488c-9fa0-9a5585f4f3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:21.041503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02501599-af78-488c-9fa0-9a5585f4f3ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:21.053280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02501599-af78-488c-9fa0-9a5585f4f3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-02501599-af78-488c-9fa0-9a5585f4f3ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:21.063613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:21.064591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 17:28:21.065408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:21.068742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 245/479] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:21.074848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:21.197932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d982-2c4c-47d8-abac-c96058593793 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:21.244388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f533e-2690-4ead-8a28-aa6b55358cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:21.245370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f533e-2690-4ead-8a28-aa6b55358cab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:21.245370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f533e-2690-4ead-8a28-aa6b55358cab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:21.245644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f533e-2690-4ead-8a28-aa6b55358cab None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:21.246575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f533e-2690-4ead-8a28-aa6b55358cab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:21.246892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f533e-2690-4ead-8a28-aa6b55358cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:21.248369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f533e-2690-4ead-8a28-aa6b55358cab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:21.262053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 235/480] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:22.259892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441858c3-6e60-4a41-831e-7b545d92452d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:22.260349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441858c3-6e60-4a41-831e-7b545d92452d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:22.260565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441858c3-6e60-4a41-831e-7b545d92452d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:22.262157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 246/481] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:22.267253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36827e-bfcf-4a58-bf3f-76a1a5e16980 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:22.267647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36827e-bfcf-4a58-bf3f-76a1a5e16980 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:22.267830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36827e-bfcf-4a58-bf3f-76a1a5e16980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:22.290506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8a36827e-bfcf-4a58-bf3f-76a1a5e16980 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:22.291736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8a36827e-bfcf-4a58-bf3f-76a1a5e16980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:22.295122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c57b-05ab-4c5d-8763-5948c969264b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:22.296140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c57b-05ab-4c5d-8763-5948c969264b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:22.296140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c57b-05ab-4c5d-8763-5948c969264b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:22.299089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 247/482] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:22.304856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f188cb-c773-4c64-86fb-358afdb7232c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:22.305152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f188cb-c773-4c64-86fb-358afdb7232c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:22.305423 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f188cb-c773-4c64-86fb-358afdb7232c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:22.330250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:22.338018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-25f188cb-c773-4c64-86fb-358afdb7232c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:22.339236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-25f188cb-c773-4c64-86fb-358afdb7232c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:22.376950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 236/483] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:22.404670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36827e-bfcf-4a58-bf3f-76a1a5e16980 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:22.461890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 248/484] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:22.497239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f188cb-c773-4c64-86fb-358afdb7232c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:22.512968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:22.582653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36827e-bfcf-4a58-bf3f-76a1a5e16980 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:22.621506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:22.622892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:22.622892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:22.623060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:22.641086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:22.658412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:22.659344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:22.660124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:22.661236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 237/485] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:22.667690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:22.694782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f188cb-c773-4c64-86fb-358afdb7232c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:22.701111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:22.767037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:22.767764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:22.768050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:22.768329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:22.789235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:22.801047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:22.801916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:22.802683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 249/486] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:22.809799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:22.824371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e48cc-5b23-4f5d-a12a-c61054d4be21 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:22.890967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:22.893046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:22.893542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:22.893825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:22.894590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:22.894900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:22.896301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:22.916259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 238/487] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:22.921745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:22.940975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:22.996893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a80b4-a8f7-4f32-ae44-c0621321ba52 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.031839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:23.042068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168537f-17ad-49ef-b40c-a75d2afcd930 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.081322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.082134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.082423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.082707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.090447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.092972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.093259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.093536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.094405 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:23.094649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.096058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.108629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.109155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.111646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 250/488] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:23.116851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:23.148786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:23.149682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:23.150512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:23.150512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:23.150512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:23.150512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:23.153086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 239/489] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:23.158624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.237379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c78b0-3566-41cb-aa1e-369139bbe87c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.277287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:23.293964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.293964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.293964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.293964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.301616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.303886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:23.322835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:23.323714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:23.324725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:23.324725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:23.324725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:23.324725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:23.325140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675067a1-0a70-410a-803e-0b291ee5dbed None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.325778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 251/490] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:23.332550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.389291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b3b19-1480-40b3-886d-897ec5eb5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.389956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b3b19-1480-40b3-886d-897ec5eb5214 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.390238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b3b19-1480-40b3-886d-897ec5eb5214 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.390512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b3b19-1480-40b3-886d-897ec5eb5214 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.391282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b3b19-1480-40b3-886d-897ec5eb5214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:23.391586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b3b19-1480-40b3-886d-897ec5eb5214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.392997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b3b19-1480-40b3-886d-897ec5eb5214 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.409999 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 240/491] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:23.411391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91292c-13f6-40ea-9caa-666bd95ec477 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.411679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91292c-13f6-40ea-9caa-666bd95ec477 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.411954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91292c-13f6-40ea-9caa-666bd95ec477 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.413494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 241/492] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:23.414195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91292c-13f6-40ea-9caa-666bd95ec477 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.422362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:23.462139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290706d-0865-478d-9871-c6c439f3dfe7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.512148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe6afc-f045-4b1e-ac06-328d734b07b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.513187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe6afc-f045-4b1e-ac06-328d734b07b5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.513187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe6afc-f045-4b1e-ac06-328d734b07b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.513385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe6afc-f045-4b1e-ac06-328d734b07b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.514096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe6afc-f045-4b1e-ac06-328d734b07b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:23.514402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe6afc-f045-4b1e-ac06-328d734b07b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.515766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe6afc-f045-4b1e-ac06-328d734b07b5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.529691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 252/493] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:23.530493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91292c-13f6-40ea-9caa-666bd95ec477 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.531151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78daf4-5123-46ba-97c7-f1141d15bf01 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.531473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78daf4-5123-46ba-97c7-f1141d15bf01 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.535587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78daf4-5123-46ba-97c7-f1141d15bf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.555520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ed78daf4-5123-46ba-97c7-f1141d15bf01 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:23.556904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ed78daf4-5123-46ba-97c7-f1141d15bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:23.591352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.592678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.593539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.593989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.604156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.605418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:23.607997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:23.630252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:23.631392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 17:28:23.632503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:23.634610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 242/494] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:23.635573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.641975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 253/495] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:23.642925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78daf4-5123-46ba-97c7-f1141d15bf01 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.700916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:23.740789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c948e09-79a3-4ffc-aedb-7f7300ced309 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.764437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78daf4-5123-46ba-97c7-f1141d15bf01 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.777352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dde41-58e2-48ec-a8e1-1ef760d9334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.778252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dde41-58e2-48ec-a8e1-1ef760d9334d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.778534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dde41-58e2-48ec-a8e1-1ef760d9334d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.778919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dde41-58e2-48ec-a8e1-1ef760d9334d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.786986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dde41-58e2-48ec-a8e1-1ef760d9334d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.788729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dde41-58e2-48ec-a8e1-1ef760d9334d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.799366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.800065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.800353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.800653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.801498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:23.801808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:23.803266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3e8dde41-58e2-48ec-a8e1-1ef760d9334d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:23.803782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:23.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 243/496] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11b735-4c20-42dd-a0be-5098273c128c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.811035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11b735-4c20-42dd-a0be-5098273c128c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.811877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11b735-4c20-42dd-a0be-5098273c128c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.811877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 244/497] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:23.811877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11b735-4c20-42dd-a0be-5098273c128c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.852172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 254/498] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:23.852172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:23.889923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:23.942711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11b735-4c20-42dd-a0be-5098273c128c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.970376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:23.976254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20db37-6dfd-44b8-8153-1c64c8ba66bb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:23.982249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f9c92-d1e9-4ea6-aea1-e78882e7874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:23.983058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9c92-d1e9-4ea6-aea1-e78882e7874f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:23.983345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9c92-d1e9-4ea6-aea1-e78882e7874f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:23.983748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9c92-d1e9-4ea6-aea1-e78882e7874f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:23.996879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f9c92-d1e9-4ea6-aea1-e78882e7874f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-019f9c92-d1e9-4ea6-aea1-e78882e7874f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:24.006187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:24.007100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:24.007872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 245/499] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:24.009022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2783a8-6a64-453d-bb26-f7956a0f9407 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.009022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2783a8-6a64-453d-bb26-f7956a0f9407 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.009195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2783a8-6a64-453d-bb26-f7956a0f9407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.016251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.016961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.017253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.017535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.018388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:24.018709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.020167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.026226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8b2783a8-6a64-453d-bb26-f7956a0f9407 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:24.027649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8b2783a8-6a64-453d-bb26-f7956a0f9407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:24.038560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 255/500] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:28:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:24.039392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.100282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 246/501] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:24.101158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2783a8-6a64-453d-bb26-f7956a0f9407 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.144951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.154976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddaf62-588e-4f39-b0cb-2eff24b91177 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.189448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c11f2-6434-40a2-9ab4-9ce80cb6e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.190094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c11f2-6434-40a2-9ab4-9ce80cb6e56d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.190380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c11f2-6434-40a2-9ab4-9ce80cb6e56d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.190658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c11f2-6434-40a2-9ab4-9ce80cb6e56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.200533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2783a8-6a64-453d-bb26-f7956a0f9407 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.215983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.246849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.247564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.247856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.248142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.248944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:24.249264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.252040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.263255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 247/502] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:24.264079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.285234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c11f2-6434-40a2-9ab4-9ce80cb6e56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.286718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c11f2-6434-40a2-9ab4-9ce80cb6e56d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.363984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3216a2-5255-4460-b6f6-cc0d43894770 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.375951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.385895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 256/503] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:24.387190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fe744-aba6-48f3-b40b-9ecd39fe2bbf None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.387481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fe744-aba6-48f3-b40b-9ecd39fe2bbf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.387760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fe744-aba6-48f3-b40b-9ecd39fe2bbf None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.391673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 257/504] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:24.391673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fe744-aba6-48f3-b40b-9ecd39fe2bbf None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.400058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.400988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.401279 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.401567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.443197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.468953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.470995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.491574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fe744-aba6-48f3-b40b-9ecd39fe2bbf None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.525611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.526683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.526969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.527249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.538335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:24.545993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:24.547017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:24.547824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 258/505] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:24.548645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.574349 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 248/506] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:24.575181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.599474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.648852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.649359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77b5f3-b073-439b-a8ce-0db26155d6e5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.673629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da44e4b-04e8-45b4-9360-4f9982436872 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.683896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c1601-7db6-43dd-93f0-7a2d8a6ef5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.684717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c1601-7db6-43dd-93f0-7a2d8a6ef5ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.685005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c1601-7db6-43dd-93f0-7a2d8a6ef5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.685287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c1601-7db6-43dd-93f0-7a2d8a6ef5ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.696172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c1601-7db6-43dd-93f0-7a2d8a6ef5ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.697648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c1601-7db6-43dd-93f0-7a2d8a6ef5ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.708252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.709062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.709348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.709627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-4f7c1601-7db6-43dd-93f0-7a2d8a6ef5ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:24.714907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:24.715816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:24.716693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:24.716693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:24.716693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:24.716693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:24.716693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 259/507] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:24.717274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65537b97-dccd-44b3-93cf-b63c7d82f4a9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.717579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65537b97-dccd-44b3-93cf-b63c7d82f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.717844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65537b97-dccd-44b3-93cf-b63c7d82f4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.721365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.722854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.734878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-65537b97-dccd-44b3-93cf-b63c7d82f4a9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:24.736085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-65537b97-dccd-44b3-93cf-b63c7d82f4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:24.740996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:24.742003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:24.743042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:24.743042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:24.743042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:24.743042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:24.743042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 249/508] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:24.743350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.805081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 260/509] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:24.805880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65537b97-dccd-44b3-93cf-b63c7d82f4a9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.837957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.842162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd4ca0-b9b4-4089-8bb8-34049404a6ab None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.867840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:24.876370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.877063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.877360 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.877641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.878393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:24.878704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.880093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.888428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 250/510] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:24.889689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.910029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65537b97-dccd-44b3-93cf-b63c7d82f4a9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:24.943932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:24.944783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:24.945074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:24.945352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:24.955545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:24.957053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:24.974961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:24.976028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:24.976957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:24.976957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:24.976957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:24.976957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:24.976957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 261/511] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:24.976957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:24.992344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e5a34-a478-4b89-af50-4f89051c4b19 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.025092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.026020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.026662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.026662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.027296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:25.027604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.029027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.040701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 251/512] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:25.041747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.060311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.076624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98c2b-b062-4247-b09e-5fa03f375c7c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.091849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.114118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.114784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.115061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.115336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.116061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:25.116366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.117794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.130521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 262/513] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:25.131365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.153003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623aa52-eef9-4c74-b2a7-2bcff45147c2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.186892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.187590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.187880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.188767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.200925 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.202722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:25.219487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:25.220473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:25.221364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:25.221364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:25.221364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:25.221364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:25.221364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 252/514] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:25.221909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.252716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d5bff-73c1-4edc-8efd-589edf567b85 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.291965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.336183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.336183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.336183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.336183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.336183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.336183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:25.336818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:25.337761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:25.338774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:25.338774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:25.338774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:25.338774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:25.338774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 263/515] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:25.338774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.339166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be611230-b121-4647-ac2d-4d4277b14a25 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.339166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.366680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.367427 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.367631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.367908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.368666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:25.368978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.370386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.386626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 253/516] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:25.386626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.438980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9243e16-c390-4b1f-b9b2-306fe5cef649 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.475238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.476073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.476349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.476656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.484549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.486609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.493969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ada1dc-7de6-41ad-84e0-cd3f49f5e829 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:25.503114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:25.504357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 264/517] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:25.509986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.532657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.533466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.533750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.534026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.541692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.543180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:25.561926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:25.562853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:25.563727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 254/518] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:25.564191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.570705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.586226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.616535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bebb1-e566-4f38-a0bf-49ba556ad29e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.651274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.651928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.652208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.652479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.653233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:25.653532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.654909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.666572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3a642-ac47-47c7-810c-e4105ce1654f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.666943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 265/519] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:25.667742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.704099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.704721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.705010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.705291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.706014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:25.706319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.707707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.719173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 255/520] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:25.720481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.765206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5ad52-7f40-4e59-ba0f-aca0c436cb75 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.781376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.799217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.799749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:25.800002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.800286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.800561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.807237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.808740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.818351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7c087-9511-4f7c-8259-9c5b7f0c6f93 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:25.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:25.823491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:25.824309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 266/521] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:25.824810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.855982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.856683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.856973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.857252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.858094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:25.858415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.859882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.895494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 256/522] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:25.896298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.931241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffc936-fe68-4ab5-b69e-3dc99c38fdd0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:25.965584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:25.966172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:25.966457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:25.966735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:25.967521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:25.967832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:25.969878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:25.987154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 267/523] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:25.992628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:25.998076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b486b1-17d1-4b1f-89bc-3cc44e4ddad7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.027297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:26.032875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.033536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.033829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.034109 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.034820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.036339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.044826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:26.090409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53456a1b-a573-4619-be03-c34bc0524a7b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.123264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.124055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.124338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.124634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.131426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.132886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:26.147514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:26.148570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 268/524] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:26.149691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.177225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 257/525] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:28:25 2022] POST /identity/v3/users => generated 247 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:26.177225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.266665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:26.273024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca3520-943e-41ad-9de1-86e1703c07fe None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.296216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee5212-b8e5-4e7a-a760-dce2d7957cb4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.314777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.315448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.315734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.316015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.316843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:26.317157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.318616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.337310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 269/526] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:26.337310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.344214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.344214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.344214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.344214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.417783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:26.425795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.427094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.450534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f319a-d871-4538-9291-5ab563b5069a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.487526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.488492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.488836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.489273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.490110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=b3b0f154abf64d57a431b956a1822300) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:26.490431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.491876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.497705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 270/527] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/role_assignments?user.id=b3b0f154abf64d57a431b956a1822300&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:26.498245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.525499 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:26.563753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 258/528] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:26.563753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.631418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d10654-3798-4c30-b446-9e1181779c29 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.675471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.676516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.677018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.677489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.678540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:26.679042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.680983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.686810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dda943-aca2-4804-b797-6cc1388ab88f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.701512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:26.705583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 271/529] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:26.706770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.723203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.723869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.724156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.724437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.725431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=3ba11eb5837a4567bd77f3fcb0017ea7) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:26.725747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.727271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.733659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 259/530] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/role_assignments?user.id=3ba11eb5837a4567bd77f3fcb0017ea7&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:26.734543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.827409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:26.874506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c14495-a67c-4193-9b19-308c6e28e783 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.913772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c50fe2-095f-44f3-8d73-037e6e888d25 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:26.923489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.924294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.924581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.924886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:26.936811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:26.938895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:26.958906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:26.959819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:26.960691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:26.960691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:26.960691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:26.960691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:26.960691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 272/531] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:26.968765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:26.970049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:26.970449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:26.970928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:26.971420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:27.001742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:27.066545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:27.068663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:27.184072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb732d8-f5e6-493e-bb6d-ef4b3c75ff82 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:27.200648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 260/532] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:27.200648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:27.255270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:27.264040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:27.264770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:27.275935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:27.275935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:27.275935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:27.275935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:27.275935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:27.276298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:27.293851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 273/533] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:27.299074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:27.317976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3041d5-3bae-4606-ae94-bbf526881007 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:27.365477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:27.366335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:27.366621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:27.366898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:27.386189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:27.388724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:27.413658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:27.414557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:27.415384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:27.415384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:27.415384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:27.415384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:27.415384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 261/534] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:27.420480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:27.510772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a838a-27dc-40a4-a89b-1c9b81ffe422 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:27.528146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269e207-d213-4737-93bc-9b1bfdb27f31 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:27.549112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:27.568288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:27.568288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:27.568288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:27.568288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:27.569087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:27.569087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:27.569087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:27.588438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 262/535] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:27.594508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:27.637314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:27.638137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:27.638422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:27.638699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:27.653689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:27.655155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:27.684209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:27.685186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:27.686027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:27.697724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 274/536] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:27.699427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:27.738954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52fb29-635e-4268-970e-a6d7e770c853 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:27.776158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:27.776920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:27.777214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:27.777496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:27.793706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:27.795195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:27.798980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:27.824631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:27.825539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:27.826384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:27.828964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 263/537] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:27.834642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:27.854198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccb498-3141-41d2-bc7a-494be507143d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:27.899813 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:27.900489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:27.901061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:27.901344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:27.902138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:27.902450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:27.903913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:27.917600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 275/538] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:27.920960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:27.928588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:28.043645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382b9f3-cfe6-487d-8af3-ce2d608bcc0e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:28.091797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14163f05-fd3d-49d7-a632-cf92861f1df8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:28.102189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:28.102883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.103176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.103464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.104298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:28.104631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:28.105592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:28.106170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:28.132272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da830df-9cb1-4cba-9532-7189a55238ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:28.133085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da830df-9cb1-4cba-9532-7189a55238ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.133367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da830df-9cb1-4cba-9532-7189a55238ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.133643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da830df-9cb1-4cba-9532-7189a55238ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.134600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da830df-9cb1-4cba-9532-7189a55238ee None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=4c1a2fa417e1456ba4c36d6e4dc4014d) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:28.134912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da830df-9cb1-4cba-9532-7189a55238ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:28.136413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da830df-9cb1-4cba-9532-7189a55238ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:28.140897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 264/539] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:28.146535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:28.148717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 276/540] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:27 2022] GET /identity/v3/role_assignments?user.id=4c1a2fa417e1456ba4c36d6e4dc4014d&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:28.322701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:28.336863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187fd73-afdc-43c8-89e0-7e8d8a94e9a9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:28.394728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c816c9-93b8-410f-883e-c570a5f9b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:28.395392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c816c9-93b8-410f-883e-c570a5f9b843 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.395677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c816c9-93b8-410f-883e-c570a5f9b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.395953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c816c9-93b8-410f-883e-c570a5f9b843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.396839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c816c9-93b8-410f-883e-c570a5f9b843 None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=35d3862fe9e846f18d8bc3a0fa5e993a) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:28.397156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c816c9-93b8-410f-883e-c570a5f9b843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:28.398624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c816c9-93b8-410f-883e-c570a5f9b843 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:28.404955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 265/541] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:28 2022] GET /identity/v3/role_assignments?user.id=35d3862fe9e846f18d8bc3a0fa5e993a&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:28.602040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1439bb0-998c-4f41-a224-3463b41a3e8e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.603467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1439bb0-998c-4f41-a224-3463b41a3e8e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.604011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1439bb0-998c-4f41-a224-3463b41a3e8e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.606156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 277/542] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:28.613395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7482d-0292-49ee-a7b8-b049a0cb4560 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.614058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7482d-0292-49ee-a7b8-b049a0cb4560 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.614058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7482d-0292-49ee-a7b8-b049a0cb4560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.637724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e3a7482d-0292-49ee-a7b8-b049a0cb4560 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:28.638975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e3a7482d-0292-49ee-a7b8-b049a0cb4560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:28.720477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 266/543] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:28.751116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1439bb0-998c-4f41-a224-3463b41a3e8e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:28.889096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1439bb0-998c-4f41-a224-3463b41a3e8e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:28.926115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:28.926926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.927216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.927499 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.941606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1706d-5064-41a9-85e4-261e0e1fe0e4 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.941911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1706d-5064-41a9-85e4-261e0e1fe0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.942497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1706d-5064-41a9-85e4-261e0e1fe0e4 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.943682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:28.944736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 267/544] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:28.954646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c772d-b2d5-415a-85a3-73532baff443 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:28.955011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c772d-b2d5-415a-85a3-73532baff443 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:28.955225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c772d-b2d5-415a-85a3-73532baff443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:28.956640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:28.957727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:28.958525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 278/545] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:28.965699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.004692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:29.005612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1d6c772d-b2d5-415a-85a3-73532baff443 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:29.006849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1d6c772d-b2d5-415a-85a3-73532baff443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:29.106347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 268/546] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:29.122828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a94c1-7b9c-44ee-8ea5-ead3d01ddbde None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.129376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c772d-b2d5-415a-85a3-73532baff443 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.145799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:29.163400 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7cc04-61a1-4b3f-9c6e-534a0415941e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.164023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7cc04-61a1-4b3f-9c6e-534a0415941e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.164311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7cc04-61a1-4b3f-9c6e-534a0415941e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.164589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7cc04-61a1-4b3f-9c6e-534a0415941e None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.165386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7cc04-61a1-4b3f-9c6e-534a0415941e None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:29.165693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7cc04-61a1-4b3f-9c6e-534a0415941e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:29.167141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7cc04-61a1-4b3f-9c6e-534a0415941e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:29.181393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 279/547] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:29.181393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c28c-ecf7-41a8-9a77-c40c55d740c6 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.181393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c28c-ecf7-41a8-9a77-c40c55d740c6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.181393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c28c-ecf7-41a8-9a77-c40c55d740c6 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.181393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 280/548] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:29.182716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641c28c-ecf7-41a8-9a77-c40c55d740c6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.256061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c772d-b2d5-415a-85a3-73532baff443 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.287223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:29.294768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641c28c-ecf7-41a8-9a77-c40c55d740c6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.296743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526b0ae-0bc2-4ad6-bca3-7ee2b375b7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.297885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526b0ae-0bc2-4ad6-bca3-7ee2b375b7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.298349 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526b0ae-0bc2-4ad6-bca3-7ee2b375b7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.298794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526b0ae-0bc2-4ad6-bca3-7ee2b375b7f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.310237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526b0ae-0bc2-4ad6-bca3-7ee2b375b7f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-4526b0ae-0bc2-4ad6-bca3-7ee2b375b7f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:29.318798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:29.319693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:29.321006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:29.330195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 269/549] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:29.331913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af896c-0248-40c8-a9e8-d91164389e9c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.332364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af896c-0248-40c8-a9e8-d91164389e9c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.332965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af896c-0248-40c8-a9e8-d91164389e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.333426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.334373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.334739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.335107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.347501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:29.348898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:29.353458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-16af896c-0248-40c8-a9e8-d91164389e9c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:29.354900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16af896c-0248-40c8-a9e8-d91164389e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:29.367912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:29.369169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:29.370034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:29.370034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:29.370034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:29.370034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:29.373836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 281/550] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:29.374664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.431849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 270/551] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:29.433219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af896c-0248-40c8-a9e8-d91164389e9c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.436754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:29.484827 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d92f-f301-494a-9005-408db61c42c7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.522331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.523445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.523445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.523856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.524433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:29.524796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:29.530038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:29.547562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 282/552] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:29.547562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.554281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af896c-0248-40c8-a9e8-d91164389e9c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.559642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:29.615989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.617355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.617642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.617922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.618656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:29.618964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:29.620370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:29.636148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 271/553] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:29.641915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.680552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e69708-419f-4991-9c7b-3aa47a8aa09e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.711960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:29.724481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.727430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.727430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.727430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.737261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:29.745420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:29.746533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:29.747419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:29.748023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 283/554] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:29.749332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.763457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f03ee1-a95c-498e-ae8c-52756acb9652 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.808309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.809703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.810146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.810430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.821679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:29.823307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:29.829353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:29.841719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:29.842699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:29.843577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:29.843577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:29.843577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:29.843577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:29.847840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 272/555] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:29.848860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.869009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec94a4-9813-4d23-93c0-d64560447679 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:29.911997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:29.913325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:29.913908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:29.914472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:29.923286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:29.925049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:29.941167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:29.942175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:29.943457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:29.950953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 284/556] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:29.952234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:29.981956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80516c38-797e-41d9-ae76-a6ce1c722c2a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.001435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.026613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.027283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.027568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.027846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.028729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:30.040566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.040566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.040566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 273/557] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:30.040829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.068272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c6041-2c3f-428a-849a-520076535cdb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.099556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.106388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.114841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.114841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.114841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.114841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:30.114841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.114841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.125119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 285/558] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:30.126893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.154720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24e936-982c-408d-b493-eb11c568f990 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.191963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.193060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.193348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.193627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.194751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:30.195063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.197189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.225933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 274/559] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:30.225933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.245194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5386b70-1769-4bbf-83cb-f6d0b80cf36d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.277307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.286717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.288248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.288929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.289314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.305318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.307766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:30.332833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:30.333864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:30.334757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:30.334757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:30.334757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:30.334757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:30.334984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 286/560] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:30.336374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.361461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6013c1-2eb3-4f3e-87dc-4cbe2c853297 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.399207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.407213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.408386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.408841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.409537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.419696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.422146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:30.442849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:30.444170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 17:28:30.445259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:30.450680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 275/561] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:30.450680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.483309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dc824-6312-4dd4-8908-90f00dcc7486 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.527709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9e75b-4e89-4347-8039-5b34410b7193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.528286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9e75b-4e89-4347-8039-5b34410b7193 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.528628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9e75b-4e89-4347-8039-5b34410b7193 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.528890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9e75b-4e89-4347-8039-5b34410b7193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.564198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.570468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262015f-4481-40ed-b2b0-e2f0a3232e22 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.602398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9e75b-4e89-4347-8039-5b34410b7193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.603885 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9e75b-4e89-4347-8039-5b34410b7193 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.609152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.610417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.610704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.610984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.611714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:30.612091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.614196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.634450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 276/562] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:30.635313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.648366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.712844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 287/563] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:30.714198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb9f97-22fe-4bb8-84cb-dbd248193001 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.714490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb9f97-22fe-4bb8-84cb-dbd248193001 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.714771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb9f97-22fe-4bb8-84cb-dbd248193001 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.716211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 288/564] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:30.721392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb9f97-22fe-4bb8-84cb-dbd248193001 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.745962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888c4e2-fc12-4460-8b03-3b9e497d662f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.782712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e241b3-8182-4100-b61b-fc3ebc70c3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.783412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e241b3-8182-4100-b61b-fc3ebc70c3ac None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.783699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e241b3-8182-4100-b61b-fc3ebc70c3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.783974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e241b3-8182-4100-b61b-fc3ebc70c3ac None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.784852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e241b3-8182-4100-b61b-fc3ebc70c3ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:30.785167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e241b3-8182-4100-b61b-fc3ebc70c3ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.786617 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e241b3-8182-4100-b61b-fc3ebc70c3ac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.806216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 277/565] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:30.808716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3c10b-4aed-4a7e-bb04-3ff77a6b06fe None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.809248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3c10b-4aed-4a7e-bb04-3ff77a6b06fe None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.809927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3c10b-4aed-4a7e-bb04-3ff77a6b06fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.827715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bbf3c10b-4aed-4a7e-bb04-3ff77a6b06fe None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:30.829577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbf3c10b-4aed-4a7e-bb04-3ff77a6b06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:30.831139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb9f97-22fe-4bb8-84cb-dbd248193001 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:30.839511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.865413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:30.866210 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:30.866501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:30.866783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:30.873960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:30.875429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:30.889818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:30.890793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:30.892290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 289/566] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:30.892901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.898182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:30.909388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 278/567] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:30.909388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3c10b-4aed-4a7e-bb04-3ff77a6b06fe None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:30.996417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7fb6b-c971-4431-83cf-1fc6b9c3164b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.008417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3c10b-4aed-4a7e-bb04-3ff77a6b06fe None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.031278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f0fb6-0f3d-45c5-a55b-db15d74603e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.032792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0fb6-0f3d-45c5-a55b-db15d74603e4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.033028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0fb6-0f3d-45c5-a55b-db15d74603e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.033312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0fb6-0f3d-45c5-a55b-db15d74603e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.042434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.043214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.043494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.043610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f0fb6-0f3d-45c5-a55b-db15d74603e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.043764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.044981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f0fb6-0f3d-45c5-a55b-db15d74603e4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.053743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.055192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.057773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ad1f0fb6-0f3d-45c5-a55b-db15d74603e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:31.062141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:31.063251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:31.064342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:31.064342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:31.064342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:31.064342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:31.064342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 290/568] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:31.065046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3712e-0f2e-40db-b20a-82e8f3042c12 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.065336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3712e-0f2e-40db-b20a-82e8f3042c12 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.065624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3712e-0f2e-40db-b20a-82e8f3042c12 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.068640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 291/569] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:31.068640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3712e-0f2e-40db-b20a-82e8f3042c12 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:31.079669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:31.080701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:31.081601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:31.081601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:31.081601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:31.081601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:31.081601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 279/570] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:31.081879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.133207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.178566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3712e-0f2e-40db-b20a-82e8f3042c12 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.184539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f40330-8b71-4fff-a42e-782b2b4e27f6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.213493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.214161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.214448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.214732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.215567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:31.215880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.217364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.219652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540edb99-b5e0-489a-8ea9-5e5ae784abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.221216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540edb99-b5e0-489a-8ea9-5e5ae784abe0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.221502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540edb99-b5e0-489a-8ea9-5e5ae784abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.221779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540edb99-b5e0-489a-8ea9-5e5ae784abe0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.232039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540edb99-b5e0-489a-8ea9-5e5ae784abe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:31.237192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 292/571] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:31.238016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-540edb99-b5e0-489a-8ea9-5e5ae784abe0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:31.239812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:31.241114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:31.241956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 280/572] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:31.242658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf04fa-cc39-40e3-a213-3aeb644c7630 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.243088 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf04fa-cc39-40e3-a213-3aeb644c7630 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.243248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf04fa-cc39-40e3-a213-3aeb644c7630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.259892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d9bf04fa-cc39-40e3-a213-3aeb644c7630 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:31.261111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d9bf04fa-cc39-40e3-a213-3aeb644c7630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:31.310958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.328208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 281/573] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:31.329131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf04fa-cc39-40e3-a213-3aeb644c7630 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.350243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06887be-8b54-4234-8b4d-5b9405de3ea8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.364733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.385715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.386378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.386667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.386948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.387677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:31.387986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.389389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.401588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 293/574] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:31.402023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.429139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf04fa-cc39-40e3-a213-3aeb644c7630 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.463377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.464041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.464330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.464629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.465361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:31.465671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.467071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.478596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 282/575] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:31.479391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.505978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23261165-b19d-4cb1-a219-97c70c26b76b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.527741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.540325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.541155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.541476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.541989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.552835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.554276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:31.571125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:31.572019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:31.572939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:31.572939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:31.572939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:31.572939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:31.572939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 294/576] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:31.573259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.583894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538a07c-e61c-4998-adb7-b17429b8bc03 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.598958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.617350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.618029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.618317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.618600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.619347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:31.619712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.621071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.628994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 283/577] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:31.629783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.676581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059ac04-f27a-4c1e-8145-747606a44123 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.710412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.711207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.711494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.711778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.721900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:31.729680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:31.730802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:31.731807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:31.732572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde9aca-0200-46fa-ae1e-96184d488890 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.733034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 295/578] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:31.733034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.741717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.767499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.768319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.768623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.768908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.775526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.777682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:31.791534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:31.798137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 284/579] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:31.799262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.821440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.836858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b94af1-8074-4c8c-80c5-10e8c5ea0dfe None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.870394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.871182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.871468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.871754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.879021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.880516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:31.895070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:31.896087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:31.902048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:31.902584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b13d31-6176-42ba-af42-3bc8c3dc0d9a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:31.902667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 296/580] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:31.902667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.933697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:31.934375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:31.934661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:31.934940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:31.935664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:31.935972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:31.937410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:31.949149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 285/581] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:31.949952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:31.981155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:31.999607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7cf0-7ddb-4b65-b895-63f86758d176 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.041426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.045733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.046042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.048084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.061608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f83a08-b569-4688-b8a9-805e2fb9e1e4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.072429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.073907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.076696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:32.097562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:32.098709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:32.099604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:32.099604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:32.099604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:32.099604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:32.099604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 297/582] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:32.099894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.106672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.107347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.107634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.107912 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.108695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:32.109007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.111033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.119524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 286/583] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:32.120665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.219773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:32.222718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b2848-9004-4100-8531-4d5e77386972 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.229948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ac46d-7a98-4feb-a641-49199c12815b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.262715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.263386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.263672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.264099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.266458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:32.266788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.271494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.276710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.277390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.277678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.277962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.278844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:32.279259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.280753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.296649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 287/584] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:32.304069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 298/585] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:32.304211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.304301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.371265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:32.417735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e343855-b86c-4850-a003-da939c7a0f8c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.418472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6476b9-1ca9-4be3-8806-8af80ef27f33 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.450471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.451276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.451559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.451841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.455278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.455938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.456222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.456498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.457257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:32.457564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.458923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.459011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.460413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.471694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 299/586] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:32.472530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:32.475911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:32.477108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 288/587] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:32.477997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.490162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:32.578054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c54ee-e0de-412b-9c17-a07f623bd30a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.579099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be3dc-e081-4b89-a4bc-28c5ef06e54b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.595755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:32.614535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.615432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.615713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.615880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.616093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.616485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.616810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.617104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.617948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=7a49603273fe4eff892e8b024a473158) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:32.618419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.620066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.626815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 289/588] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/role_assignments?user.id=7a49603273fe4eff892e8b024a473158&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:32.627567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.647065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.648563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:32.668297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:32.669549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:32.670645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:32.670645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:32.670645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:32.670645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:32.670645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 300/589] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:32.670645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.736910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:32.771992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719897a-ee86-4362-b784-e9290c53410f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.788261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49296-4b02-4d0a-9701-c6f0e7a1dd6c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:32.819571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.820477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.820800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.821067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/b3b0f154abf64d57a431b956a1822300/roles/22cdd113399a479bb54fcb86fc3f2a15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.832319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:32.833159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:32.833445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:32.833723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:32.840408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=b3b0f154abf64d57a431b956a1822300, role_id=22cdd113399a479bb54fcb86fc3f2a15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.841417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:32.842090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.842855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:32.857585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:32.858714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 301/590] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:32.860188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.888350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:32.924753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 290/591] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:32 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/b3b0f154abf64d57a431b956a1822300/roles/22cdd113399a479bb54fcb86fc3f2a15 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:32.925798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:32.982553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825194a-064f-4f17-8fc3-754972420559 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.031733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.033485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.034178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.034633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.035632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:33.036119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.037878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.051167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1ece-c000-4c07-9cd8-aa08557bdb51 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.052181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 302/592] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:33.053391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.083131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:33.098131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.098813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.099101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.099380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.100122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:33.100434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.102552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.121221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 291/593] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:33.122989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.171235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:33.187416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43753a64-4638-4240-a5f0-5fd087230737 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.238080 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.238764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.239050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.239195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0726-7ffd-4908-b890-ad7b9ccb9c38 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.239332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.240184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:33.240497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.242276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.266324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 303/594] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 15 17:28:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:33.267481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.278836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.279843 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.280193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.280539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/3ba11eb5837a4567bd77f3fcb0017ea7/roles/22cdd113399a479bb54fcb86fc3f2a15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.299590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=3ba11eb5837a4567bd77f3fcb0017ea7, role_id=22cdd113399a479bb54fcb86fc3f2a15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.301262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.408868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 292/595] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:33 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/3ba11eb5837a4567bd77f3fcb0017ea7/roles/22cdd113399a479bb54fcb86fc3f2a15 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:33.409994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.442632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ccbf0-2779-462d-98a5-f6cff4070c83 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.480445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:33.513451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.514287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.514575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.514850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.534105 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.535597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.565706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4045be9-f7ba-49da-b3fa-8393e65e5cc9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:33.574612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:33.575708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:33.576729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 304/596] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:33.581490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.601574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:33.617985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.618792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.619080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.619358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.635649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.637220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:33.661475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:33.662437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:33.663448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:33.663448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:33.663448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:33.663448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:33.663448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 293/597] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:33.668837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.745322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fea0-30a9-4211-a752-04a1a3c8620d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.771269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:33.788672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d429c1-e85b-4adb-bcb5-93e7ba8b718e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:33.828865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.829755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.830052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.830338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.831095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:33.831409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.832908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.847214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 294/598] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:33.852715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.853352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:33.854050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:33.854339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:33.854620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:33.855797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:33.856115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:33.858646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:33.879884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 305/599] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Jul 15 17:28:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:33.885332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:33.969524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:33.999936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96377f15-8c3c-4ed4-8c47-b774ce23ba86 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:34.042026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:34.043103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:34.043493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:34.043873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:34.055620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:34.057401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:34.077878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:34.079578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:34.080582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:34.080582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:34.080582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:34.080582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:34.081750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 306/600] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:34.087360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:34.116190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a444f-8e9b-4f82-81ba-4e67ccd58894 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:34.183757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:34.188279 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:34.188863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:34.189390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:34.199342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:34.201107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:34.220114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:34.221245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:34.222325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:34.237591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 295/601] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:34.248731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:34.269129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:34.287497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a5eff-b52f-472a-99a0-dd901664aca7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:34.302007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:34.334236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:34.335035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:34.335432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:34.335776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:34.338025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:34.338413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:34.340738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:34.354956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 307/602] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:34.360267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:34.471734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da270347-6a93-4b89-9eb7-047a719a68fb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:34.494777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088d70b-3f42-4a47-b3cf-7bd7ac87665a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:34.508051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:34.509095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:34.509485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:34.509860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:34.517177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:34.518831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:34.534459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:34.535531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:34.536526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:34.537715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 308/603] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:34.546541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:34.571106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:34.571826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:34.572159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:34.572447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:34.573278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:34.575145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:34.579023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:34.604030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 296/604] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:34.623422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:34.629006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:34.739969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6d609-5ac5-4ca8-a25b-384a4ecddd2b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:34.758398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:34.790475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:34.791536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:34.793122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:34.793639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:34.794751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:34.795371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:34.798015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:34.820981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 309/605] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:34.826952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:34.840409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8a57f-e906-44ac-98fa-c6663b8e87f8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:34.910244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126b4f5-06c5-4ab0-a001-9193dae178c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:34.911523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126b4f5-06c5-4ab0-a001-9193dae178c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:34.912022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126b4f5-06c5-4ab0-a001-9193dae178c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:34.912513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126b4f5-06c5-4ab0-a001-9193dae178c4 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/4c1a2fa417e1456ba4c36d6e4dc4014d/roles/22cdd113399a479bb54fcb86fc3f2a15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:34.932163 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126b4f5-06c5-4ab0-a001-9193dae178c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=4c1a2fa417e1456ba4c36d6e4dc4014d, role_id=22cdd113399a479bb54fcb86fc3f2a15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:34.934213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126b4f5-06c5-4ab0-a001-9193dae178c4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:34.987772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:34.994240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dec2f-8d4b-4566-b5b1-e31b61c541ee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:35.029331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441a55c-91a9-47a3-b31d-69ac9d8207fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:35.030252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441a55c-91a9-47a3-b31d-69ac9d8207fd None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.030541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441a55c-91a9-47a3-b31d-69ac9d8207fd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.030823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441a55c-91a9-47a3-b31d-69ac9d8207fd None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/35d3862fe9e846f18d8bc3a0fa5e993a/roles/22cdd113399a479bb54fcb86fc3f2a15` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.045781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441a55c-91a9-47a3-b31d-69ac9d8207fd None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=35d3862fe9e846f18d8bc3a0fa5e993a, role_id=22cdd113399a479bb54fcb86fc3f2a15)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:35.047358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441a55c-91a9-47a3-b31d-69ac9d8207fd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:35.052962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 297/606] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:34 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/4c1a2fa417e1456ba4c36d6e4dc4014d/roles/22cdd113399a479bb54fcb86fc3f2a15 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:28:35.056071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1548a1-d372-49d9-8608-e63ea7d586ff None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.061024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1548a1-d372-49d9-8608-e63ea7d586ff None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.061024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1548a1-d372-49d9-8608-e63ea7d586ff None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.061024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 298/607] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:35.065238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b41d4-bb32-411d-9d32-05d509325b96 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.065530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b41d4-bb32-411d-9d32-05d509325b96 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.065967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b41d4-bb32-411d-9d32-05d509325b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.085004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-676b41d4-bb32-411d-9d32-05d509325b96 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:35.086291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-676b41d4-bb32-411d-9d32-05d509325b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:35.144991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 310/608] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:34 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/35d3862fe9e846f18d8bc3a0fa5e993a/roles/22cdd113399a479bb54fcb86fc3f2a15 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:35.146711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc809b0a-323a-4d9f-affc-3fff6d440776 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.147013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc809b0a-323a-4d9f-affc-3fff6d440776 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.147384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc809b0a-323a-4d9f-affc-3fff6d440776 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.149232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 311/609] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:35.162974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e49738-e38f-4db0-a352-a90dcb458a3a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.163377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e49738-e38f-4db0-a352-a90dcb458a3a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.163616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e49738-e38f-4db0-a352-a90dcb458a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.167504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 299/610] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:35.192336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b41d4-bb32-411d-9d32-05d509325b96 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:35.195229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a0e49738-e38f-4db0-a352-a90dcb458a3a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:35.196572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a0e49738-e38f-4db0-a352-a90dcb458a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:35.248632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:35.286731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 312/611] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:35.310058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b41d4-bb32-411d-9d32-05d509325b96 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:35.311044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e49738-e38f-4db0-a352-a90dcb458a3a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:35.354096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:35.354921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.355361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.355466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.371381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:35.386581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:35.387584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:35.388934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 300/612] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:35.395029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:35.408201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:35.435742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e49738-e38f-4db0-a352-a90dcb458a3a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:35.475624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:35.476434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.476752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.477034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.488443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:35.496507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:35.497488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:35.498766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 313/613] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:35.504199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:35.509038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdcc11-4d60-4dbe-9a4e-a7f744843d4b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:35.517022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:35.547958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:35.548626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.548935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.549217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.549982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:35.550296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:35.551711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:35.560329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 301/614] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:35.569605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:35.618194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c29131-e857-44df-b8d6-4ec925623558 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:35.655897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:35.656666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.656964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.657249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.658010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:35.658330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:35.659857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:35.669426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 314/615] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:35.670081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:35.678089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:35.804460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e9c0e-c3dd-4ad4-a3b9-66be52802259 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:35.862041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2e20-2689-43de-ac73-149ca7bfb222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:35.863570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2e20-2689-43de-ac73-149ca7bfb222 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.864067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2e20-2689-43de-ac73-149ca7bfb222 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.864516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2e20-2689-43de-ac73-149ca7bfb222 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.868177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70df7fb-0bd7-4b5d-8d83-ef0965d490cb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:35.876723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e2e20-2689-43de-ac73-149ca7bfb222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:35.878431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e2e20-2689-43de-ac73-149ca7bfb222 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8d2e2e20-2689-43de-ac73-149ca7bfb222 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:35.896526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:35.897442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:35.898306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:35.898306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:35.898306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:35.898306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:35.905117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 302/616] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:35.907103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d227-4fca-426e-a004-5eaad93d34de None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.907684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d227-4fca-426e-a004-5eaad93d34de None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.908235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d227-4fca-426e-a004-5eaad93d34de None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.909224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d83d9-8b3a-446d-a949-45692331ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:35.910160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d83d9-8b3a-446d-a949-45692331ac51 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.910554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 303/617] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:35.910656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d83d9-8b3a-446d-a949-45692331ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.911500 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d227-4fca-426e-a004-5eaad93d34de None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:35.911923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d83d9-8b3a-446d-a949-45692331ac51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.930239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d83d9-8b3a-446d-a949-45692331ac51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:35.932094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d83d9-8b3a-446d-a949-45692331ac51 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:35.937098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-641d83d9-8b3a-446d-a949-45692331ac51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:35.953093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:35.960509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 315/618] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b87187-dad6-4ab1-aa68-7ec237ba6f8c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b87187-dad6-4ab1-aa68-7ec237ba6f8c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:35.961409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b87187-dad6-4ab1-aa68-7ec237ba6f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:35.982682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f8b87187-dad6-4ab1-aa68-7ec237ba6f8c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:35.984207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f8b87187-dad6-4ab1-aa68-7ec237ba6f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:36.050936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.052831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d227-4fca-426e-a004-5eaad93d34de None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.080753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 316/619] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:36.082055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b87187-dad6-4ab1-aa68-7ec237ba6f8c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.102042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.103062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.103569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.104995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.105949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:36.107008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.108750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.122515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 304/620] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:36.128444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.212383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.229197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b87187-dad6-4ab1-aa68-7ec237ba6f8c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.237423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09508-ceb1-428a-a6b4-f9ed671c55ed None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.271431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.272403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.272929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.273415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.274372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:36.274873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.276765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.284003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.285412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.285712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.285992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.295590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 317/621] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:36.299540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.303139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:36.313137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:36.314061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:36.314867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:36.316678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 305/622] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:36.317768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.360705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.427584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db40c5-c1b2-4ed5-bf6d-4a5b70d8f44a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.431775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98d13d-c81a-47b0-a5f4-e1aab389e7ee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.463389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.464212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.464501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.464811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.469469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.470384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.470680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.470959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.474321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.475812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.478814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.480196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.481741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:36.491868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:36.497782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 306/623] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:36.498711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:36.499355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:36.500713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 318/624] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:36.504819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.606201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6aafc-2db2-4153-a300-16687d0bed77 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.612522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.616099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bebe4-97d4-47d3-b7af-e1c6eb640b53 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.642119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.643173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.643664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.644170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.645176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:36.645708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.647280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.651148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.651825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.652110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.653856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.655137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:36.655631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.657962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.658395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 307/625] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:36.659398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.681378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 319/626] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:36.682283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.722797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.762266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515045c7-3ce2-4e8f-98f8-1500400a97a6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.789327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795637a-3af8-44e0-8774-830e307edb07 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.799012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.799997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.800507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.801021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.802082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:36.802595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.804248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.823117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 308/627] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:36.824357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.826580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.827428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.827867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.828180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.839382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.840930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:36.859364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:36.860963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:36.861964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:36.861964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:36.861964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:36.861964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:36.861964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 320/628] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:36.866399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:36.881348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.932778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce9d03-7fdc-43d7-92cf-936382ff4f1e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.955020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:36.970191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:36.971256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:36.971708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:36.972142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:36.978616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce46ad-0e9b-476d-8084-25bb9eb73cb4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:36.982990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:36.985199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:37.004442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:37.005574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:37.006612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:37.006612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:37.006612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:37.006612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:37.008411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 309/629] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:37.009498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.016987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.017643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.017927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.018300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.096222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.097733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.109862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.116548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4018e-65a4-40fd-bd5a-6ebd48ae5991 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.153313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.154354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.154850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.155335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.156814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:37.157383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.159001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.175469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 310/630] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:37.176655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.208197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 321/631] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:37.209042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.209696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.283388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10595e-73f4-44f9-a8bc-fddb2e6ae307 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.322788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.323829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.324362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.324869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.325807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:37.326316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.327925 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.344440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cee2e6-1da1-4dd3-945b-498eb085631e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.345011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 311/632] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:37.346129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.357707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.387518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.388322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.388614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.388911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.396062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.398396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:37.413225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:37.414337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:37.415335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:37.418908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 322/633] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:37.419740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.443024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.451497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ae41-3fbf-44f5-8781-e76269833950 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.486284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5017103-0c38-4382-9fd6-8b81e059a3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.487012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017103-0c38-4382-9fd6-8b81e059a3bc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.487297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017103-0c38-4382-9fd6-8b81e059a3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.487576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017103-0c38-4382-9fd6-8b81e059a3bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.497270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5017103-0c38-4382-9fd6-8b81e059a3bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.498721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5017103-0c38-4382-9fd6-8b81e059a3bc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c5017103-0c38-4382-9fd6-8b81e059a3bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:37.516417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:37.517520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:37.518401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:37.518401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:37.518401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:37.518401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:37.518401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 312/634] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:37.518678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acd9f3-5122-487a-9e76-1045ed9ed532 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.519108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bda76-4cb4-4c9a-b91e-8ee6ab23da33 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.519108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bda76-4cb4-4c9a-b91e-8ee6ab23da33 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.519387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bda76-4cb4-4c9a-b91e-8ee6ab23da33 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.520646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 313/635] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:37.521707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f1e3-f27e-423a-a3f9-7670944cfbda None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.522212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f1e3-f27e-423a-a3f9-7670944cfbda None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.522311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f1e3-f27e-423a-a3f9-7670944cfbda None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.523452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 314/636] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:37.524135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788f1e3-f27e-423a-a3f9-7670944cfbda None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.559807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.560592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.560917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.561194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.567807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.569273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:37.583627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:37.584786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 323/637] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:37.585782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.616010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.628283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788f1e3-f27e-423a-a3f9-7670944cfbda None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.664703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc47c5-386f-43bd-b5da-13e4d5b98dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.665374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc47c5-386f-43bd-b5da-13e4d5b98dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.665660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc47c5-386f-43bd-b5da-13e4d5b98dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.665937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc47c5-386f-43bd-b5da-13e4d5b98dd4 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.666783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc47c5-386f-43bd-b5da-13e4d5b98dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:37.667093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc47c5-386f-43bd-b5da-13e4d5b98dd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.668544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc47c5-386f-43bd-b5da-13e4d5b98dd4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.685413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 315/638] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:37.685982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a3b21-33e7-4bf5-86cc-390e5eb95095 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.686644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccae88-f9c9-41fd-a84b-a9e79bf62597 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.686933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccae88-f9c9-41fd-a84b-a9e79bf62597 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.687210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccae88-f9c9-41fd-a84b-a9e79bf62597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.692065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.703900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cdccae88-f9c9-41fd-a84b-a9e79bf62597 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:37.705135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cdccae88-f9c9-41fd-a84b-a9e79bf62597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:37.720231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd34e64-c795-4f0f-a856-2df2c42201e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.721005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd34e64-c795-4f0f-a856-2df2c42201e5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.721307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd34e64-c795-4f0f-a856-2df2c42201e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.721572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd34e64-c795-4f0f-a856-2df2c42201e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.722297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd34e64-c795-4f0f-a856-2df2c42201e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:37.722612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd34e64-c795-4f0f-a856-2df2c42201e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.723971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd34e64-c795-4f0f-a856-2df2c42201e5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.735764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 324/639] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:37.737103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea529b-02db-4e90-bfcf-d2d15548d220 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.737346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea529b-02db-4e90-bfcf-d2d15548d220 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.737623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea529b-02db-4e90-bfcf-d2d15548d220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.755407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-daea529b-02db-4e90-bfcf-d2d15548d220 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:37.756627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-daea529b-02db-4e90-bfcf-d2d15548d220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:37.776020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 316/640] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:37.785789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccae88-f9c9-41fd-a84b-a9e79bf62597 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.823746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 325/641] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:37.824606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea529b-02db-4e90-bfcf-d2d15548d220 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.845825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.890502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccae88-f9c9-41fd-a84b-a9e79bf62597 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.919711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:37.924452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.925144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.925434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.925714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:37.926508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:37.926823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:37.927182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea529b-02db-4e90-bfcf-d2d15548d220 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:37.928289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:37.945061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 317/642] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:37.945061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:37.961407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:37.962055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:37.962340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:37.962618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.031758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.033196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.041034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537830f1-b29f-408a-8ba9-ee2191bbf84d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.053581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:38.077875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.079245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.079531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.079843 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.087111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.088589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:38.103425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:38.104556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:38.105565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:38.106009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 318/643] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:38.106368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.156399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 326/644] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:38.157385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.163705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:38.211009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5c9a-d928-4c2c-9be0-b3dcc814fe4d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.246574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.247379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.247670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.247951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.258754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:38.259343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239864b6-16a4-4f51-b5ce-efd8f42c379a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:38.266488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:38.267469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:38.268446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 319/645] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:38.269205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.284036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:38.298268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.299063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.299348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.299632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.309485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:38.318189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:38.319307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:38.320249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 327/646] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:38.321043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.371111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da1df5-fccd-4404-ad09-12bca190c3f0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.398116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:38.419388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.420277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.420564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.420878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/7a49603273fe4eff892e8b024a473158/roles/22cdd113399a479bb54fcb86fc3f2a15` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.420974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf37d1-d57b-48db-a4b6-49a1682f8646 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.435862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=7a49603273fe4eff892e8b024a473158, role_id=22cdd113399a479bb54fcb86fc3f2a15)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.437351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.455534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.456191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.456476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.456789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.457568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:38.457875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.459292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.471058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 328/647] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:38.471916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.507541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 320/648] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:38 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/7a49603273fe4eff892e8b024a473158/roles/22cdd113399a479bb54fcb86fc3f2a15 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:38.508568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.520470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:38.590502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc15f2-11ab-4fcc-99e0-0401306b3261 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.629128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.629940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.630222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.630495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.633079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda40fd9-bff3-4fb4-8bd4-1328c2a6d15e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.641434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.642924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:38.666733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:38.667660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:38.668529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:38.668529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:38.668529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:38.668529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:38.668529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 329/649] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:38.670501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.677622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.679250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.679250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.679250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.679711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:38.680014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.681559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.692087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 321/650] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:38.693085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.720855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:38.768320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:38.777410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41992-467e-437c-a57f-96b8ee0cec91 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.802781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e023e7-5f14-4333-93dd-c5688d545c60 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.815417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.816088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.816373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.816673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.817418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:38.817721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.819099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.832080 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 330/651] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:38.832934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.841167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:38.842189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:38.842472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:38.842745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:38.843583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=b3b0f154abf64d57a431b956a1822300) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:38.844823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:38.846716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:38.853961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 322/652] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/role_assignments?user.id=b3b0f154abf64d57a431b956a1822300&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:38.854775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:38.959603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5b59-ac0b-4534-80e6-6d96afd8deb1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:38.988412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:39.005610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632247-b792-47f9-953b-91454cbf24bb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.011994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.013020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.013518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.014012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.015484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:39.016505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.018920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.034313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 331/653] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:39.035177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.050525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.051345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.051625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.051905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.057821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:39.069437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.070915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:39.091591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:39.092557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:39.093534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:39.093534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:39.093534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:39.093534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:39.095043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 323/654] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:39.095219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.179767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56726-3a0b-4b48-9e42-ee4f0cc26061 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.213098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956508-418e-40a0-84b5-9f0d69bf1b61 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.230133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.230948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.231535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.231816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.244370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.245822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.262421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.263577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.264090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.264566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:39.268903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:39.269801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:39.270732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:39.270732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:39.270732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:39.270732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:39.270732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 332/655] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:39.271420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.276982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.278755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.282317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:39.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:39.299327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:39.300218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:39.301187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 324/656] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:39.302147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.369051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:39.397165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66760fa-7729-4208-8398-e0f16e70dcac None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.432616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51295d60-3bef-4175-bc83-f0f454b9b2ff None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.444493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.445354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.445741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.446121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.446964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:39.447355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.449026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.463446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 333/657] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:39.463446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.478454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.479518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.479965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.480384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.481874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:39.482333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.484390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.504698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 325/658] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:39.504698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.571583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:39.592365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394037-ec64-4927-92a6-3dcc6457a56c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.613070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e736-bdc4-43ac-bd89-8e00c2dd976e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.627182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:39.639223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.640503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.641467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.641920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.643725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:39.648492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.648492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.654634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.655514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.655873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.656765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.664679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 334/659] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:39.664828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.665913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.666973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:39.683493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:39.684470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:39.685429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 326/660] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:39.686017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.800429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43e513-fbae-4ad2-b90e-2309023e1f83 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.802492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadd8db-d34e-4d42-8ab5-47e0221fee53 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:39.822035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:39.838044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.838801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.839170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.839525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.840430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=3ba11eb5837a4567bd77f3fcb0017ea7) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:39.840839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.841084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:39.842048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:39.842479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.842703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:39.843150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:39.850018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 327/661] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/role_assignments?user.id=3ba11eb5837a4567bd77f3fcb0017ea7&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:39.851061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.851948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:39.853615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:39.888177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:39.889240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 17:28:39.890165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:39.901858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 335/662] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:39.908975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:39.965316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66dc71-acba-489a-bd6e-6ae836f35040 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:40.003303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:40.004283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.004722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.005111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.006165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:40.006579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:40.007360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:40.009411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:40.030551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 328/663] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:40.036914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:40.091006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09d3a1-4ca5-4da1-ada4-fed31da44e49 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:40.106722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:40.143402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:40.144351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.144725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.144995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.146287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:40.146587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:40.148025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:40.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea17fa2-bd37-4cec-b3f9-802548d5e64c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:40.169831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 336/664] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Jul 15 17:28:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:40.175495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:40.216848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:40.217574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.217868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.218153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.299718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:40.301325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:40.308379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2791-47eb-4e78-9704-e77c48ed090d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:40.311361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:40.378997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:40.379194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:40.379709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.379993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.380266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.470931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 329/665] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:40.479079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:40.492325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:40.493927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:40.608784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796c0ae-00c6-4ad6-873f-5353347780c3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:40.648737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:40.649543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.649829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.650108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.654063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 337/666] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:28:40.664636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:40.666210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:40.670993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:40.673170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:40.681335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:40.689960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:40.690887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:40.691756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:40.691756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:40.691756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:40.691756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:40.691756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 330/667] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:40.697203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:40.848921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7bdea-2309-4a35-aca7-edb556099539 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:40.852102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091f5e-b632-438d-aeb1-8c9e74d0b5cd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:40.882332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:40.883137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.883526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.883810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.889649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937a141-1667-4afb-b1de-c862b9814dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:40.890599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937a141-1667-4afb-b1de-c862b9814dec None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.891070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937a141-1667-4afb-b1de-c862b9814dec None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.891527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937a141-1667-4afb-b1de-c862b9814dec None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.892428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937a141-1667-4afb-b1de-c862b9814dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:40.892973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937a141-1667-4afb-b1de-c862b9814dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:40.894566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937a141-1667-4afb-b1de-c862b9814dec None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:40.895305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:40.897083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:40.908137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 331/668] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:40.914085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86429d-d48b-4abc-8bc4-7ac179e99231 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:40.914590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86429d-d48b-4abc-8bc4-7ac179e99231 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:40.915051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86429d-d48b-4abc-8bc4-7ac179e99231 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:40.916832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 332/669] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:40.917844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86429d-d48b-4abc-8bc4-7ac179e99231 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:40.917973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:40.918767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:40.919653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:40.919653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:40.919653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:40.919653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:40.919653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 338/670] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:40.925188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:40.979599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.026000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86429d-d48b-4abc-8bc4-7ac179e99231 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.032530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7a01a-1267-4ef1-83c8-03887b2a9730 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.048974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.061426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.063104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.063814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.064297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.069642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4712a-1113-4ab5-86d7-2e8f2620a213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.070313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4712a-1113-4ab5-86d7-2e8f2620a213 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.070604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4712a-1113-4ab5-86d7-2e8f2620a213 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.070889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4712a-1113-4ab5-86d7-2e8f2620a213 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.071681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4712a-1113-4ab5-86d7-2e8f2620a213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:41.071998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4712a-1113-4ab5-86d7-2e8f2620a213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.073462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4712a-1113-4ab5-86d7-2e8f2620a213 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.073875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.075535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.091421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 339/671] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:41.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:41.092751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:41.093622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:41.094589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8c44-2b3f-43ab-acef-a1487ac5fecc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.094589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8c44-2b3f-43ab-acef-a1487ac5fecc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.094589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8c44-2b3f-43ab-acef-a1487ac5fecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.101456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 333/672] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:41.101989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.116613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dd5f8c44-2b3f-43ab-acef-a1487ac5fecc None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:41.117840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dd5f8c44-2b3f-43ab-acef-a1487ac5fecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:41.188344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 340/673] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:41.189247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f8c44-2b3f-43ab-acef-a1487ac5fecc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.213674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0e355-9704-4da7-8f7f-4db5c4399806 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.233864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.237682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.258243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.259068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.259500 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.259785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.267334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.269369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:41.284154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:41.285207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:41.286091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:41.288909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 334/674] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:41.290164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.298485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f8c44-2b3f-43ab-acef-a1487ac5fecc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.335023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18db65-e7c2-4af9-9b28-a8ccb1595597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.335738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18db65-e7c2-4af9-9b28-a8ccb1595597 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.335974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18db65-e7c2-4af9-9b28-a8ccb1595597 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.336262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18db65-e7c2-4af9-9b28-a8ccb1595597 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.337095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18db65-e7c2-4af9-9b28-a8ccb1595597 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:41.337407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18db65-e7c2-4af9-9b28-a8ccb1595597 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.339030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18db65-e7c2-4af9-9b28-a8ccb1595597 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.353645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 341/675] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:41.354901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328d056-7cdf-4308-85dc-c179f136be07 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.355197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328d056-7cdf-4308-85dc-c179f136be07 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.355476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328d056-7cdf-4308-85dc-c179f136be07 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.360844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 342/676] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:41.360844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328d056-7cdf-4308-85dc-c179f136be07 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.405827 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb28263-9e25-42f3-8eba-8f5ed0a4a815 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.439746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.440546 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.440862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.441142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.451667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:41.459836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:41.460965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:41.462008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:41.465801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 335/677] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:41.466642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.476202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328d056-7cdf-4308-85dc-c179f136be07 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.484844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.503066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.513435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c21315-b932-4393-8cc4-2cb56b7ab487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.514094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c21315-b932-4393-8cc4-2cb56b7ab487 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.514379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c21315-b932-4393-8cc4-2cb56b7ab487 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.514652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c21315-b932-4393-8cc4-2cb56b7ab487 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.515441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c21315-b932-4393-8cc4-2cb56b7ab487 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:41.516450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c21315-b932-4393-8cc4-2cb56b7ab487 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.518094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c21315-b932-4393-8cc4-2cb56b7ab487 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.537164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 343/678] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:41.537164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309c15-8344-40c6-9750-567ada3aaf96 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.537164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309c15-8344-40c6-9750-567ada3aaf96 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.537164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309c15-8344-40c6-9750-567ada3aaf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.550439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2c309c15-8344-40c6-9750-567ada3aaf96 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:41.551632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2c309c15-8344-40c6-9750-567ada3aaf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:41.581315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143d19-e4a1-41eb-9a76-56b6fe5f6fe3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.617718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.618303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.618588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.618875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.619689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=4c1a2fa417e1456ba4c36d6e4dc4014d) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:41.620006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.621421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 344/679] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:41.621530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.622540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c309c15-8344-40c6-9750-567ada3aaf96 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.634221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 336/680] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/role_assignments?user.id=4c1a2fa417e1456ba4c36d6e4dc4014d&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:41.634221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.745011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.749234 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368ab7b-5249-4a1b-8f18-4721b63ae0cf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.751806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:41.763467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c309c15-8344-40c6-9750-567ada3aaf96 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.791952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.792822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.793207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.793583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.794659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=35d3862fe9e846f18d8bc3a0fa5e993a) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:41.795488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.797965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.798532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:41.799222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:41.799512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:41.799793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:41.800688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:41.801006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:41.802532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:41.806090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 337/681] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/role_assignments?user.id=35d3862fe9e846f18d8bc3a0fa5e993a&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:41.807111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.818159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 345/682] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:41.824565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:41.962515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2fe82-990e-493f-bc7f-0141087c25a9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:41.963025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1066dd7-4203-4283-8570-839b19c34a33 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.011963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.013004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.013436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.013806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.014092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.014372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.014552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.015017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.029082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.032271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.032472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.033855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:42.054224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:42.055147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:42.056064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:42.056064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:42.056064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:42.056064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:42.056291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:42.057138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:42.058034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:42.058034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:42.058034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:42.058034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:42.058247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 346/683] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:42.058314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 338/684] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:42.061243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.061876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.079893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:42.087847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:42.195754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cc87b-2ae4-49fe-94e3-dac249bbb038 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.245097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.245732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.248629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.248629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.248629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:42.248629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.248629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.262458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 347/685] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:42.268344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.294665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcdfaf-cdeb-467b-81d4-6a07e1498d67 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.330319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ad9d8-07bc-494a-b8fa-0ae36739e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.331942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ad9d8-07bc-494a-b8fa-0ae36739e40f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.331942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ad9d8-07bc-494a-b8fa-0ae36739e40f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.331942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ad9d8-07bc-494a-b8fa-0ae36739e40f None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.332652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ad9d8-07bc-494a-b8fa-0ae36739e40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:42.333039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ad9d8-07bc-494a-b8fa-0ae36739e40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.334518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ad9d8-07bc-494a-b8fa-0ae36739e40f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.346474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:42.347693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 339/686] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:42.349183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174b2f8-ca59-4afb-b79c-4d5eca4d1684 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.349477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174b2f8-ca59-4afb-b79c-4d5eca4d1684 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.349759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174b2f8-ca59-4afb-b79c-4d5eca4d1684 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.351040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 340/687] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:42.355959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174b2f8-ca59-4afb-b79c-4d5eca4d1684 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.380916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2f417-0d9c-4a80-a28e-08bf3ecdc84a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.433516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fcca5-f81b-4910-852b-00cec2f8aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.434999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fcca5-f81b-4910-852b-00cec2f8aa56 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.435294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fcca5-f81b-4910-852b-00cec2f8aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.435577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fcca5-f81b-4910-852b-00cec2f8aa56 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.446297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fcca5-f81b-4910-852b-00cec2f8aa56 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.447768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fcca5-f81b-4910-852b-00cec2f8aa56 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.448859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-535fcca5-f81b-4910-852b-00cec2f8aa56 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:42.471069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:42.472047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:42.473010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:42.476707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 348/688] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:42.478217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c4336-d9ee-44ac-afed-cefe6c20ac78 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.478968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c4336-d9ee-44ac-afed-cefe6c20ac78 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.479773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c4336-d9ee-44ac-afed-cefe6c20ac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.489467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174b2f8-ca59-4afb-b79c-4d5eca4d1684 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.503501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ab7c4336-d9ee-44ac-afed-cefe6c20ac78 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:42.505799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ab7c4336-d9ee-44ac-afed-cefe6c20ac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:42.527232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.527906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.528237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.528557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.529308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.530763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.586119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 349/689] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:42.587306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c4336-d9ee-44ac-afed-cefe6c20ac78 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.608797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 341/690] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:28:42 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:42.617284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.662766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:42.712478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c4336-d9ee-44ac-afed-cefe6c20ac78 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.727734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29a94f-0d39-41b1-bf28-37eed8a51d62 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.753773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.754838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.755341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.755890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.757235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:42.757798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.759440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:42.760004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.765212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.766063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.766344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.766664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.780065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:42.783370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 350/691] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:42.790126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:42.792741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:42.793672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:42.794493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 342/692] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:42.800846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.919436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e6a4a-ff0c-4a5d-ae2f-3c3ff4826edb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.964295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:42.965159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:42.965444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:42.965722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:42.966778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:42.967090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:42.968924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:42.970050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024e38-2a15-485b-980e-a17a5eb24be8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:42.979954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 343/693] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:42.986289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:42.995900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:43.017749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.017749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.017749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.017749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:43.046368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:43.090662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:43.092466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:43.096653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a51a1-8fa4-4f68-837b-c12b2bb9d134 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:43.133256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.134062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.134346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.134624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:43.148693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:43.150185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:43.169021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:43.169964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:43.170835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:43.170835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:43.170835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:43.170835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:43.170835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 344/694] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:43.176068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:43.207665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 351/695] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 17:28:43.216739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:43.238732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:43.297508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af588b17-2e60-473b-9d38-9b3bd1e2d300 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:43.303044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:43.334471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.335137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.335424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.335702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:43.336428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:43.336865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:43.338278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:43.350211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 345/696] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:43.355677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:43.440940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8b06-778f-4bae-9d5d-f90b52261d73 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:43.472590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de52530-d501-457b-b065-8561a234ca40 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:43.508823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.509629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.509918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.510367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:43.519181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:43.521049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:43.526139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.526949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.527234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.527511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:43.536782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:43.537757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:43.538715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 346/697] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:28:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:43.543304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:43.543817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:43.544887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:43.563004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:43.564876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:43.566002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:43.566002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:43.566002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:43.566002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:43.566002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 352/698] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:43.574403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:43.576972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:43.700754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a83b9-12d0-46a7-93af-b6f060900297 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:43.714979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:43.759262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322b2ca-b399-47d3-827f-ef9c5be8c1af None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:43.773160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.773851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.774140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.774441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:43.775306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:43.775941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:43.777464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:43.814632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 347/699] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:28:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:43.819915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:43.833500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.834523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.835016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.835495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:43.836482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:43.837086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:43.838715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:43.852671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 353/700] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:43.867580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:43.913204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:43.928082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614be1c-ec6b-4aea-871d-facbde41baa2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:43.963444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737385a3-cab6-4b4b-bb0c-dafb0b905353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:43.964107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737385a3-cab6-4b4b-bb0c-dafb0b905353 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:43.964395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737385a3-cab6-4b4b-bb0c-dafb0b905353 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:43.964700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737385a3-cab6-4b4b-bb0c-dafb0b905353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.079728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06f17-3c99-44bc-84ce-73cc4310e8db None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:44.092870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737385a3-cab6-4b4b-bb0c-dafb0b905353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.094490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737385a3-cab6-4b4b-bb0c-dafb0b905353 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.129124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:44.135603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e69fda-f179-4f5a-be32-9fd9a9e42f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:44.136410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69fda-f179-4f5a-be32-9fd9a9e42f09 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.136722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69fda-f179-4f5a-be32-9fd9a9e42f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.137000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69fda-f179-4f5a-be32-9fd9a9e42f09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.145278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e69fda-f179-4f5a-be32-9fd9a9e42f09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.149525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e69fda-f179-4f5a-be32-9fd9a9e42f09 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b6e69fda-f179-4f5a-be32-9fd9a9e42f09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:44.167227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:44.168342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:44.169601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:44.176619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 354/701] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:44.176619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9318-9744-4365-91f5-b760ad4ade6d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.176619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9318-9744-4365-91f5-b760ad4ade6d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.176619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9318-9744-4365-91f5-b760ad4ade6d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.177653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 355/702] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:44.180668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9318-9744-4365-91f5-b760ad4ade6d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:44.232820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:44.261741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 348/703] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:28:44.261741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88853b-b784-40bc-bd0b-4249cd838925 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.261741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88853b-b784-40bc-bd0b-4249cd838925 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.261741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88853b-b784-40bc-bd0b-4249cd838925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.280547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2e88853b-b784-40bc-bd0b-4249cd838925 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:44.281857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2e88853b-b784-40bc-bd0b-4249cd838925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:44.333073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9318-9744-4365-91f5-b760ad4ade6d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:44.356099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 349/704] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:44.356987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88853b-b784-40bc-bd0b-4249cd838925 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:44.375011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023ad75-dda2-4423-8417-660379e3b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:44.376846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ad75-dda2-4423-8417-660379e3b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.377143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ad75-dda2-4423-8417-660379e3b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.377439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023ad75-dda2-4423-8417-660379e3b9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.378365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ad75-dda2-4423-8417-660379e3b9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:44.378682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ad75-dda2-4423-8417-660379e3b9a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.380147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023ad75-dda2-4423-8417-660379e3b9a9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.392728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 356/705] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:44.393926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63ff21-3615-4a53-98df-90baff8e3fd7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.394276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63ff21-3615-4a53-98df-90baff8e3fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.394557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63ff21-3615-4a53-98df-90baff8e3fd7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.399829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 357/706] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:44.399829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63ff21-3615-4a53-98df-90baff8e3fd7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:44.468359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:44.475512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88853b-b784-40bc-bd0b-4249cd838925 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:44.489866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:44.513805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:44.514609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.514899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.515224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.527363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.528876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.529837 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63ff21-3615-4a53-98df-90baff8e3fd7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:44.547823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:44.548768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:44.549634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:44.549634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:44.549634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:44.549634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:44.549634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 350/707] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:44.554180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:44.572219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac9559-6eb5-4174-b4da-c46b26133d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:44.573068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac9559-6eb5-4174-b4da-c46b26133d5b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.573358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac9559-6eb5-4174-b4da-c46b26133d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.573637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac9559-6eb5-4174-b4da-c46b26133d5b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.574295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac9559-6eb5-4174-b4da-c46b26133d5b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.575670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac9559-6eb5-4174-b4da-c46b26133d5b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-bdac9559-6eb5-4174-b4da-c46b26133d5b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:44.590226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:44.591149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:44.592055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 358/708] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:44.593310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc19d6-e7af-4f4f-846e-e8dcca76997e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.593310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc19d6-e7af-4f4f-846e-e8dcca76997e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.593310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc19d6-e7af-4f4f-846e-e8dcca76997e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.610674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b6dc19d6-e7af-4f4f-846e-e8dcca76997e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:44.611947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b6dc19d6-e7af-4f4f-846e-e8dcca76997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:44.675417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47c217-18d8-4df8-a20c-47190a9eff70 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:44.685083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 359/709] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:44.685935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc19d6-e7af-4f4f-846e-e8dcca76997e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:44.728427 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:44.729025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.729309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.729587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.730415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=7a49603273fe4eff892e8b024a473158) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:44.730729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.732177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.737981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:44.748865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 351/710] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/role_assignments?user.id=7a49603273fe4eff892e8b024a473158&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:44.748865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:44.791896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:44.820919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc19d6-e7af-4f4f-846e-e8dcca76997e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:44.860363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:44.861033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.861308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.861586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.862319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:44.862619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.864258 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.877838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 360/711] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:44.878776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:44.893201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a7d2f-510a-4c67-8a63-d99163fb27f5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:44.943126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:44.944035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:44.944035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:44.944398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:44.945246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:44.945584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:44.947055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:44.957544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 352/712] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:44.957544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:45.020357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455d87b-3433-481b-bec1-8261cec51135 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.024389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:45.067681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.068715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.069010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.069310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] PATH_INFO: `/v3/services/network` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.070109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:45.071583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:45.089054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:45.093245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76779bee-8816-49c6-b8a6-e76a76d6941b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:45.095223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:45.099051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:45.099921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:45.101241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 361/713] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:45.101843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:45.146118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.146865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.147177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.147526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.155076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:45.156735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:45.176048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:45.177075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 353/714] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:45.182652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:45.234980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bcb2e-5ac7-4bb2-adee-0b1014b39f84 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.279441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.280482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.280941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.281395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.282271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:45.282841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:45.284575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:45.291430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 362/715] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 15 17:28:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:45.292436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:45.292626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e3269-7ab0-4ac5-8cbc-bbbe67ad11fe None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.329242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:45.374566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.375324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.375559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.375847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.376578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] RBAC: Adding query filter params (name=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:45.376917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:45.378330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:45.393328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 354/716] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:28:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:45.403280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:45.416300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693378d1-dcb2-43cb-883e-a09f9086fc2c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.444036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:45.462702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86075b-b640-40e3-81d0-852803272f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.463836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86075b-b640-40e3-81d0-852803272f7b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.464287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86075b-b640-40e3-81d0-852803272f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.465160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86075b-b640-40e3-81d0-852803272f7b None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.466519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86075b-b640-40e3-81d0-852803272f7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:45.467000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86075b-b640-40e3-81d0-852803272f7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:45.469732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86075b-b640-40e3-81d0-852803272f7b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:45.487022 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 363/717] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:45.489266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edb2fc-e79e-48af-8251-a4dbcdbc0f6c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.489779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edb2fc-e79e-48af-8251-a4dbcdbc0f6c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.490251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edb2fc-e79e-48af-8251-a4dbcdbc0f6c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.492669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 364/718] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:45.493971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edb2fc-e79e-48af-8251-a4dbcdbc0f6c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:45.557537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc1b15-e150-49ef-a7d0-8dc16b030565 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.597186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c00db-1dc1-4815-bc77-5a9de5762a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.597861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c00db-1dc1-4815-bc77-5a9de5762a76 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.598149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c00db-1dc1-4815-bc77-5a9de5762a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.598430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c00db-1dc1-4815-bc77-5a9de5762a76 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.599149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c00db-1dc1-4815-bc77-5a9de5762a76 None admin] RBAC: Adding query filter params (type=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:45.599457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c00db-1dc1-4815-bc77-5a9de5762a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:45.600882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c00db-1dc1-4815-bc77-5a9de5762a76 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:45.607302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 355/719] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:28:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:45.608759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7991ff-b4f4-4a4c-adb1-b6d349424b63 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.609050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7991ff-b4f4-4a4c-adb1-b6d349424b63 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.609366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7991ff-b4f4-4a4c-adb1-b6d349424b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.633368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6f7991ff-b4f4-4a4c-adb1-b6d349424b63 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:45.636477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6f7991ff-b4f4-4a4c-adb1-b6d349424b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:45.639287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:45.672533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edb2fc-e79e-48af-8251-a4dbcdbc0f6c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.758867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 356/720] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:45.760453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.761156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.761439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.761717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.762566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=3ba11eb5837a4567bd77f3fcb0017ea7) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:45.762877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:45.764378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:45.777098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 365/721] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:45 2022] GET /identity/v3/role_assignments?user.id=3ba11eb5837a4567bd77f3fcb0017ea7&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:45.808249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7991ff-b4f4-4a4c-adb1-b6d349424b63 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:45.820804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:45.927906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7991ff-b4f4-4a4c-adb1-b6d349424b63 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:45.966138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:45.966947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:45.967231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:45.967506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:45.978375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:45.986773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:45.987715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:45.988567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 357/722] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:45.997720 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:46.143923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea8685-4b74-4cd9-a2e9-7fbc69139cb1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:46.184999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:46.185602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:46.185889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:46.186167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:46.186928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:46.187317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:46.188780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:46.197767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 366/723] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:46.205689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:46.327263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:46.367869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db798834-9dd7-4af9-b0a4-ea5b67a36a40 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:46.415156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:46.415957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:46.416242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:46.416517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:46.433023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:46.434516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:46.455570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:46.457777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:46.459898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:46.459898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:46.459898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:46.459898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:46.459898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 358/724] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:46.462599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:46.503185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:46.596267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fb346-108c-43bd-a383-5c0078c90c06 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:46.640444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:46.641162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:46.641451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:46.641729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:46.642470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:46.642777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:46.644148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:46.663770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 367/725] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:46.669366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:46.808404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef4a1d-ef4f-400f-9dec-c7e768cacd7a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:46.839062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:46.856246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:46.857089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:46.857373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:46.857652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:46.867870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:46.869359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:46.891128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:46.893324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:46.895674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 359/726] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:28:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:46.901856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:47.034279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:47.056128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc661-8fdf-46d4-8c48-08ca01e025ed None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:47.091786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fda044-bf70-4397-856d-3fc7ce653098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:47.092382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fda044-bf70-4397-856d-3fc7ce653098 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.092693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fda044-bf70-4397-856d-3fc7ce653098 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.092971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fda044-bf70-4397-856d-3fc7ce653098 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.094341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fda044-bf70-4397-856d-3fc7ce653098 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:47.094650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fda044-bf70-4397-856d-3fc7ce653098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:47.096085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fda044-bf70-4397-856d-3fc7ce653098 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:47.114469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 368/727] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 15 17:28:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:47.120048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:47.276121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7251ab0-65d6-4fce-80af-143f786868e9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:47.310926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:47.311737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.312131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.312561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.361259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cdb5e-2c24-498f-afd0-369032121772 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.361892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cdb5e-2c24-498f-afd0-369032121772 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.362372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cdb5e-2c24-498f-afd0-369032121772 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.364215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 369/728] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:47.369361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16aa0c-50e3-4d0a-9174-04598afda571 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.369652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16aa0c-50e3-4d0a-9174-04598afda571 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.369930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16aa0c-50e3-4d0a-9174-04598afda571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.403298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:47.404885 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9c16aa0c-50e3-4d0a-9174-04598afda571 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:47.406652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9c16aa0c-50e3-4d0a-9174-04598afda571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:47.420815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:47.422965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:47.542542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 360/729] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 17:28:47.549062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:47.562653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 370/730] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:47.606164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16aa0c-50e3-4d0a-9174-04598afda571 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:47.660428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:47.687014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42406dd-b6e7-474e-a19e-ce5269fc6de6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:47.695922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:47.740951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16aa0c-50e3-4d0a-9174-04598afda571 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:47.748233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c8702-ad21-4bfb-a682-292b4e07f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:47.749061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8702-ad21-4bfb-a682-292b4e07f190 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.749343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8702-ad21-4bfb-a682-292b4e07f190 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.749618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8702-ad21-4bfb-a682-292b4e07f190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.764304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c8702-ad21-4bfb-a682-292b4e07f190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:47.766378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c8702-ad21-4bfb-a682-292b4e07f190 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:47.788100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:47.789239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.789766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.790249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-492c8702-ad21-4bfb-a682-292b4e07f190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:47.799484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:47.800369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:47.801481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:47.801481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:47.801481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:47.801481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:47.801481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 361/731] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:47.802464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10a90-796d-4164-9add-ead3515d6388 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.802929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10a90-796d-4164-9add-ead3515d6388 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.803214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10a90-796d-4164-9add-ead3515d6388 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.804848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 362/732] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:47.805646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:47.809932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f6770-6c65-4cee-bfa7-37f15e4c342e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.810473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f6770-6c65-4cee-bfa7-37f15e4c342e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.810961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f6770-6c65-4cee-bfa7-37f15e4c342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:47.814575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:47.815521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:47.816806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 371/733] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:47.821612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:47.837105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3b1f6770-6c65-4cee-bfa7-37f15e4c342e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:47.838706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3b1f6770-6c65-4cee-bfa7-37f15e4c342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:47.919226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 363/734] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 17:28:47.920497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f6770-6c65-4cee-bfa7-37f15e4c342e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:47.940803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fb25d-ebcf-4c4b-a7eb-ccc128f8fa02 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:47.977525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:47.988947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:47.993998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:47.995458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:47.996181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:47.996705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:47.997831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:47.998300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.001264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.019319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 372/735] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:48.020764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.030935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f6770-6c65-4cee-bfa7-37f15e4c342e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.069078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.069979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.070370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.070756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.071792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:48.072018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.073905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.087913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 364/736] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:48.089107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.164115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41186f3-0d7c-4b1b-b8b8-3b027b285b2d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.203633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ccab3-9da9-47db-bba1-fed7d45d06b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.204704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ccab3-9da9-47db-bba1-fed7d45d06b4 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.205164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ccab3-9da9-47db-bba1-fed7d45d06b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.205704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ccab3-9da9-47db-bba1-fed7d45d06b4 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.206616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ccab3-9da9-47db-bba1-fed7d45d06b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.208316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ccab3-9da9-47db-bba1-fed7d45d06b4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.211474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ccde3-7708-417a-ab03-faeb0a0b1811 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.245133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:48.254596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.255392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.255681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.255957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.262683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.264156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:48.281017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:48.285124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 365/737] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:48.286033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.311652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 373/738] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:28:48 2022] POST /identity/v3/services => generated 251 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:48.313474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba283b6-a66d-4509-aa7a-1d8d6b5ac208 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.313846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba283b6-a66d-4509-aa7a-1d8d6b5ac208 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.314203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba283b6-a66d-4509-aa7a-1d8d6b5ac208 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.319149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 374/739] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:48.319149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba283b6-a66d-4509-aa7a-1d8d6b5ac208 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.375575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:48.418784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39d4f5-affd-42da-89f4-022d6cc31421 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.449516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba283b6-a66d-4509-aa7a-1d8d6b5ac208 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.460371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd95bd-90f2-42aa-979a-f26962551ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.461133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd95bd-90f2-42aa-979a-f26962551ccf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.461133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd95bd-90f2-42aa-979a-f26962551ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.461872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd95bd-90f2-42aa-979a-f26962551ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.474085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd95bd-90f2-42aa-979a-f26962551ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.476349 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd95bd-90f2-42aa-979a-f26962551ccf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.490764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.491677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.491813 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.491991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.492820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:48.493607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.494596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c9bd95bd-90f2-42aa-979a-f26962551ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:48.494817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:48.495638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:48.496524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:48.496524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:48.496524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:48.496524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:48.496524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 366/740] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:48.501890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d327378-f85d-48fe-be06-0f79a730c336 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.502190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d327378-f85d-48fe-be06-0f79a730c336 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.502475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d327378-f85d-48fe-be06-0f79a730c336 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.503794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 367/741] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:48.504883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cdc2-d218-4845-8690-f54063284a64 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.505205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cdc2-d218-4845-8690-f54063284a64 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.505458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cdc2-d218-4845-8690-f54063284a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.512632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 375/742] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:48.513521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.523367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-33c9cdc2-d218-4845-8690-f54063284a64 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:48.524552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-33c9cdc2-d218-4845-8690-f54063284a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:48.557998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:48.597655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 368/743] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:48.598840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250ce4-912d-46bc-bfa5-41d45779bf3f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.599137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250ce4-912d-46bc-bfa5-41d45779bf3f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.599422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250ce4-912d-46bc-bfa5-41d45779bf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.620660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1d250ce4-912d-46bc-bfa5-41d45779bf3f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:48.620660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1d250ce4-912d-46bc-bfa5-41d45779bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:48.622903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29646de3-2dd9-427d-a510-58744c0f4005 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.624462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:48.658201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.658869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.659153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.659432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.660170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:48.660480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.661897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.679640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 376/744] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:48.679640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.705091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 369/745] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 17:28:48.705091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250ce4-912d-46bc-bfa5-41d45779bf3f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.800198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09324d2a-978e-4e12-b589-49e356c968ff None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.814063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:48.818061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250ce4-912d-46bc-bfa5-41d45779bf3f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:48.837939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.838598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.838884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.839163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.840030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=812f104fcd164e748c3d6558228d7ef0) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:48.840308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:48.841775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:48.856960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 377/746] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:48 2022] GET /identity/v3/role_assignments?user.id=812f104fcd164e748c3d6558228d7ef0&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:48.856960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:48.868039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:48.869012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:48.869304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:48.869585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:48.882878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:48.887151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:48.892535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:48.898457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:48.899384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 370/747] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:48.900078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.000400 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc620-8d62-45e1-9e50-48134f41453e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.023775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53554-c609-4cf7-933d-e0dd9ad7421e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.045177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:49.045982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:49.046266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:49.047101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:49.056730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:49.058135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:49.060621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:49.061543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:49.062045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:49.062584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:49.063513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:49.065335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:49.074420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:49.075441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:49.076409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 378/748] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:49.083211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.141136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 371/749] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:28:48 2022] POST /identity/v3/services => generated 240 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:49.142055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.188877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:49.225914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:49.274624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870d561-2ad5-4b59-8b7f-4257c7abf509 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.312642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3aa96-1c10-406e-bc34-37fe485c7f57 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.341905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:49.342586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:49.342877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:49.343159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:49.344589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:49.344957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:49.346530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:49.353782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:49.354669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:49.355103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:49.355584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:49.356619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:49.357077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:49.357532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 379/750] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:49.358883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:49.369522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.378655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 372/751] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 15 17:28:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:49.405976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.487197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:49.513116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5867f-ea12-4447-8ccd-dcd988e35ced None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.527311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e2752-2191-49b4-a653-64e8e5d67fba None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.563091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:49.563678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:49.563961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:49.564234 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:49.573280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:49.574078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:49.574366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:49.574645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:49.587943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:49.589832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:49.598761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:49.623269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:49.625223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:49.626218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:49.626218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:49.626218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:49.626218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:49.626218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 380/752] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:49.630841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.656449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:49.658410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:49.760908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd245af-11fe-48c2-947c-43d57686e7b6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.797253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:49.798102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:49.798402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:49.798776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:49.799413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:49.799733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:49.801295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:49.819592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 381/753] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:49.826908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.832945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:49.877401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 373/754] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 17:28:49.882803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:49.913398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:49.956869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8f6a-c501-478e-9376-560369fb4326 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:49.998881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.000075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.000570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.001107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.013135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.015059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:50.039834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:50.040848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:50.042003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:50.046345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 382/755] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:28:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:50.052124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:50.079818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af2ba-a33a-4f15-85d4-2e8fe553953c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.121557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.122921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.123449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.123933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.137145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.138973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:50.162090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:50.163121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:50.164012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:50.164012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:50.164012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:50.164012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:50.166667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 374/756] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:50.172416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:50.227740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:50.260683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b119c1-cf16-4eba-9011-b8e7f7dc5d34 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.308961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:50.340618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.341233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.341524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.341802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.342657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:50.342974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.344447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.360545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662cb7-7742-4e75-9c49-292b0bf68153 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.378832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 383/757] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:50.385583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:50.404196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.407294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.409021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.410193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.411310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:50.413105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.414955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.438662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 375/758] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:50.443819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:50.550737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:50.567796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fbe61-13ce-4e02-b56f-f47f3b626366 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.577378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307627a-feaa-42f3-94ea-4aced99bcdd9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.609109 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdf64c-176d-4a5d-83d7-1343ab444119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.610291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdf64c-176d-4a5d-83d7-1343ab444119 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.610841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdf64c-176d-4a5d-83d7-1343ab444119 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.611364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdf64c-176d-4a5d-83d7-1343ab444119 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.616904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.617699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.617985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.618263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.620773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdf64c-176d-4a5d-83d7-1343ab444119 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.622541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdf64c-176d-4a5d-83d7-1343ab444119 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.629502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.630955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-80cdf64c-176d-4a5d-83d7-1343ab444119 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:50.639236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:50.640144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:50.641242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:50.643699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 376/759] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:50.645157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af33ed-8fa1-4de6-a7b3-f7650847cf92 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.645662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af33ed-8fa1-4de6-a7b3-f7650847cf92 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.646141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af33ed-8fa1-4de6-a7b3-f7650847cf92 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.647963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 377/760] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:50.656592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:50.657531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:50.658446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:50.658446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:50.658446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:50.658446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:50.658446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 384/761] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:50.658714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290a8cb-7152-4cf8-afd6-350020e3e8ea None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.658714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290a8cb-7152-4cf8-afd6-350020e3e8ea None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.658714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290a8cb-7152-4cf8-afd6-350020e3e8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.658904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:50.676494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9290a8cb-7152-4cf8-afd6-350020e3e8ea None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:50.678659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9290a8cb-7152-4cf8-afd6-350020e3e8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:50.690892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:50.771678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873557fa-3d6b-4a8a-a17d-420bf49de905 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.789890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 378/762] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:28:50.789890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290a8cb-7152-4cf8-afd6-350020e3e8ea None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:50.829634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.830313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.830596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.830869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.835481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:50.835481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.835481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.858339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 385/763] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:50.858339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:50.906531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:50.932590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290a8cb-7152-4cf8-afd6-350020e3e8ea None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.955622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:50.965852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd735580-9f3e-4500-b9b8-1abc5e83b1ce None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:50.970697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:50.971633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:50.972124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:50.972626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:50.973561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:50.974110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:50.975704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:50.988766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 379/764] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:50.989867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.007822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.008792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.008997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.009323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.009984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.011513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:51.024360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:51.025604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:51.026427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:51.031024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 386/765] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:51.031910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.103524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47b34-0c4b-4511-8744-3875dd77b8b7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.136720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.137384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.137674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.137959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.138798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=7a49603273fe4eff892e8b024a473158) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:51.139114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.141132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.142686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c76924-c682-4113-a3f2-30e38ab7dd50 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.143125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:51.147289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 380/766] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:50 2022] GET /identity/v3/role_assignments?user.id=7a49603273fe4eff892e8b024a473158&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:51.148090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.191909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee8f3b-f8d3-4f1c-869b-265e26f2ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.193461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee8f3b-f8d3-4f1c-869b-265e26f2ef53 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.193963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee8f3b-f8d3-4f1c-869b-265e26f2ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.194738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee8f3b-f8d3-4f1c-869b-265e26f2ef53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.205528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee8f3b-f8d3-4f1c-869b-265e26f2ef53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.207333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee8f3b-f8d3-4f1c-869b-265e26f2ef53 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-16ee8f3b-f8d3-4f1c-869b-265e26f2ef53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:51.236742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:51.237678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:51.238572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:51.246743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 387/767] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:51.250521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5cedb-c772-4fe6-b8d3-cb4e7cccb876 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.251017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5cedb-c772-4fe6-b8d3-cb4e7cccb876 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.251447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5cedb-c772-4fe6-b8d3-cb4e7cccb876 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.253573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 388/768] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:51.254541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5cedb-c772-4fe6-b8d3-cb4e7cccb876 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.276154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:51.298229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b421c05-52a2-420d-9637-18fb41071672 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.345971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c02d91-d698-4046-b7a3-518787ed562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.346892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c02d91-d698-4046-b7a3-518787ed562f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.347328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c02d91-d698-4046-b7a3-518787ed562f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.347787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c02d91-d698-4046-b7a3-518787ed562f None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.348512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c02d91-d698-4046-b7a3-518787ed562f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:51.348861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c02d91-d698-4046-b7a3-518787ed562f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.350270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c02d91-d698-4046-b7a3-518787ed562f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.355404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 381/769] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:51.361238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367875-d36c-4b19-9a25-3120cc30c962 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.361803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367875-d36c-4b19-9a25-3120cc30c962 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.361915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367875-d36c-4b19-9a25-3120cc30c962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.381370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5cedb-c772-4fe6-b8d3-cb4e7cccb876 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.382618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1c367875-d36c-4b19-9a25-3120cc30c962 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:51.384103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1c367875-d36c-4b19-9a25-3120cc30c962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:51.427868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.428545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.428858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.429140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.429939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:51.430253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.431705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.450152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 389/770] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:51.450152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.472509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:51.487547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 382/771] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:28:51.488765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c367875-d36c-4b19-9a25-3120cc30c962 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.563935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:51.599327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6e05c-db25-4492-91c6-9117643807e1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.625372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c367875-d36c-4b19-9a25-3120cc30c962 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.636763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.637602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.637895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.638178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.638896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:51.639208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.640636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.645771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 390/772] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:51.648222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.666697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654c7d3-214c-4384-b170-d2f67c1dfcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.667602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c7d3-214c-4384-b170-d2f67c1dfcd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.667880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c7d3-214c-4384-b170-d2f67c1dfcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.668159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c7d3-214c-4384-b170-d2f67c1dfcd0 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/3ba11eb5837a4567bd77f3fcb0017ea7/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.670158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654c7d3-214c-4384-b170-d2f67c1dfcd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=3ba11eb5837a4567bd77f3fcb0017ea7, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.672156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654c7d3-214c-4384-b170-d2f67c1dfcd0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.745793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 383/773] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:51 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/3ba11eb5837a4567bd77f3fcb0017ea7/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:51.747142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157fc2b-ca85-47da-8a94-243a2526e536 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.747439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157fc2b-ca85-47da-8a94-243a2526e536 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.747723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157fc2b-ca85-47da-8a94-243a2526e536 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.753286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 384/774] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:51.753286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157fc2b-ca85-47da-8a94-243a2526e536 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.763189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25efb6a-c368-40b0-aa0f-bfec9b34e9c6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.805776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa01dfe-c17c-4474-befc-f1f85133a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.806660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa01dfe-c17c-4474-befc-f1f85133a7cc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.806994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa01dfe-c17c-4474-befc-f1f85133a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.807277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa01dfe-c17c-4474-befc-f1f85133a7cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.816760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:51.822607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa01dfe-c17c-4474-befc-f1f85133a7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-faa01dfe-c17c-4474-befc-f1f85133a7cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:51.831995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:51.833070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:51.833904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 391/775] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:51.835392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c1ba-b2af-4566-8637-802a2aa833da None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.835687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c1ba-b2af-4566-8637-802a2aa833da None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.835968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c1ba-b2af-4566-8637-802a2aa833da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.836916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:51.856554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-89a9c1ba-b2af-4566-8637-802a2aa833da None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:51.857997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-89a9c1ba-b2af-4566-8637-802a2aa833da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:51.907590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157fc2b-ca85-47da-8a94-243a2526e536 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:51.944929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:51.945947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:51.946469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:51.946586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 392/776] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:28:51.947058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:51.947546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c1ba-b2af-4566-8637-802a2aa833da None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:51.948309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] RBAC: Adding query filter params (interface=public, service_id=265f41c5989d42a18e56bee93b107b64, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:51.948857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:51.956619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:51.964760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 385/777] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/endpoints?service_id=265f41c5989d42a18e56bee93b107b64&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:51.970613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.075995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd97ad-8f84-4c2e-93a0-f9051f8bc039 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.085117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c1ba-b2af-4566-8637-802a2aa833da None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.093967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:52.113146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.113931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.114255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.114572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.115312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.116787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:52.121623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:52.122177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 386/778] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/services => generated 772 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:52.123025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.143429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.144134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.144426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.144733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.145526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:52.145843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.147283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:52.165563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 393/779] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:52.170848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.338271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dab1ef-3b37-41d2-9f3c-a1ef6dcc957b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.345416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaaee-d9cf-4631-a297-cb93d729db17 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.379665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.380478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.380804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.381192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] PATH_INFO: `/v3/services/network` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.381863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.383269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:52.402311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.403445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.403951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.404422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:52.404527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:52.405482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:52.406331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 387/780] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:52 2022] GET /identity/v3/services/network => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:52.411943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.423941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.425753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:52.460179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:52.461166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:52.462048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:52.462048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:52.462048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:52.462048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:52.464368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 394/781] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:52.471424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.502526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:52.542967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:52.604055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838871cd-e6ee-422f-8bbb-2fb6bc766379 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.648161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa07b0-7363-4de9-ad4a-f48796a7ee23 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.666621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.667298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.667589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.667879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.668618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] RBAC: Adding query filter params (name=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:52.668932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.670335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:52.683677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 388/782] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:28:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:52.687193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.720027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.720744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.721085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.721373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.722106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:52.722415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.724171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:52.745329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 395/783] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:52.752169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.865355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd46377-1d0f-4646-84bc-6f5cdd2d2439 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.884956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:52.912239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f1794-f754-4f0b-bffb-ef26fba1864c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:52.919573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:52.936351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.937335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.937797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.938233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.939089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] RBAC: Adding query filter params (type=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:52.939539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.941207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:52.951613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 389/784] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:28:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:52.954181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:52.954998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:52.955297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:52.955583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:52.957942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:52.973537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:52.975026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:53.000184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:53.001222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:28:53.002163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:53.008402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 396/785] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:28:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:53.013342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:53.122809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dd70d-b6b0-47d1-9b07-ab543bd06d9c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:53.162265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:53.162942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.163235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.163516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.164365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:53.164710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:53.166163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:53.201049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96618a-c1cf-4733-a1cf-85bb8b05a3bf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:53.203204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 397/786] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:53.204938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:53.247565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:53.271828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:53.272558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.272923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.273207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.274019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] RBAC: Adding query filter params (interface=public, service_id=a5de5310f9f24e39821bd492e7bd265d, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:53.274336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:53.276147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:53.289470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 390/787] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:28:52 2022] GET /identity/v3/endpoints?service_id=a5de5310f9f24e39821bd492e7bd265d&interface=public®ion_id=RegionOne => generated 194 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:53.294883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:53.375011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:53.395452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac2692-ec87-473f-959c-19ad7001a04d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:53.440482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1515e8-dca9-4705-bfa7-f71d9d30c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:53.441312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1515e8-dca9-4705-bfa7-f71d9d30c6fc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.441602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1515e8-dca9-4705-bfa7-f71d9d30c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.441881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1515e8-dca9-4705-bfa7-f71d9d30c6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.452994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1515e8-dca9-4705-bfa7-f71d9d30c6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:53.454247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1515e8-dca9-4705-bfa7-f71d9d30c6fc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:53.456136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971af2a-1841-41c8-b361-85db2c8f3ad5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8e1515e8-dca9-4705-bfa7-f71d9d30c6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:53.472625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:53.473634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:53.474494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:53.474494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:53.474494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:53.474494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:53.474494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 398/788] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:53.475530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574f74f-0baa-47e6-95f3-a95d0d784fec None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.475818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574f74f-0baa-47e6-95f3-a95d0d784fec None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.476100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574f74f-0baa-47e6-95f3-a95d0d784fec None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.481741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 399/789] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:53.481741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574f74f-0baa-47e6-95f3-a95d0d784fec None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:53.506350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2356524-dd97-47bc-b0d6-9efe493245fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:53.506917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2356524-dd97-47bc-b0d6-9efe493245fe None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.507214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2356524-dd97-47bc-b0d6-9efe493245fe None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.507495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2356524-dd97-47bc-b0d6-9efe493245fe None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.508331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2356524-dd97-47bc-b0d6-9efe493245fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:53.509845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2356524-dd97-47bc-b0d6-9efe493245fe None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:53.523677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 391/790] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:53.527137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74601bc9-e10b-4d03-b70b-b7139555385e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.529199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74601bc9-e10b-4d03-b70b-b7139555385e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.529710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74601bc9-e10b-4d03-b70b-b7139555385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.561509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-74601bc9-e10b-4d03-b70b-b7139555385e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:53.562939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-74601bc9-e10b-4d03-b70b-b7139555385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:53.582809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:53.625171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574f74f-0baa-47e6-95f3-a95d0d784fec None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:53.681521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:53.683521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.684015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.684496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.685480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:53.686129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:53.688420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:53.704933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 400/791] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:53.710538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:53.712103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 392/792] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:28:53.748582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74601bc9-e10b-4d03-b70b-b7139555385e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:53.796688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:53.854617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40803da8-44f5-43f5-95f4-a6da1e7c099b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:53.876439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74601bc9-e10b-4d03-b70b-b7139555385e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:53.896193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:53.903077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:53.904431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.904927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.905597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.915722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:53.917819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:53.919233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:53.920118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:53.920478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:53.920915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:53.934475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:53.943390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:53.945173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:53.946240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 393/793] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:53.948586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:53.949566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:53.951174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 401/794] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:53.953350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:53.957486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.074911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24200f-24e3-498b-b6be-31b528a0d8ba None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.101164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:54.113478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b6e69-f0ee-4fd3-a1bf-ffb968e3b90c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.122374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299ce51-dc7b-44ad-b2fa-c11dff33635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.123199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299ce51-dc7b-44ad-b2fa-c11dff33635f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.123554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299ce51-dc7b-44ad-b2fa-c11dff33635f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.124246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299ce51-dc7b-44ad-b2fa-c11dff33635f None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.125142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299ce51-dc7b-44ad-b2fa-c11dff33635f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:54.125776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299ce51-dc7b-44ad-b2fa-c11dff33635f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:54.127642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299ce51-dc7b-44ad-b2fa-c11dff33635f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:54.144760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 402/795] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:54.146433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbeee2-e55a-460d-97b2-0b15d067229a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.146780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbeee2-e55a-460d-97b2-0b15d067229a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.147115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbeee2-e55a-460d-97b2-0b15d067229a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.148669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 403/796] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:54.151156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbeee2-e55a-460d-97b2-0b15d067229a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.159102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6fee9-f013-4db5-bd6d-2b5de59e266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.159874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6fee9-f013-4db5-bd6d-2b5de59e266f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.160242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6fee9-f013-4db5-bd6d-2b5de59e266f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.160590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6fee9-f013-4db5-bd6d-2b5de59e266f None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.161456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6fee9-f013-4db5-bd6d-2b5de59e266f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:54.161881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6fee9-f013-4db5-bd6d-2b5de59e266f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:54.163391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6fee9-f013-4db5-bd6d-2b5de59e266f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:54.176921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 394/797] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:54.178412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f684e08-4d9d-4b3c-a3f9-500d338f8c91 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.178776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f684e08-4d9d-4b3c-a3f9-500d338f8c91 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.179130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f684e08-4d9d-4b3c-a3f9-500d338f8c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.207566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3f684e08-4d9d-4b3c-a3f9-500d338f8c91 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:54.209628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f684e08-4d9d-4b3c-a3f9-500d338f8c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:54.220969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:54.288071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbeee2-e55a-460d-97b2-0b15d067229a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.292980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 395/798] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:28:54.293985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f684e08-4d9d-4b3c-a3f9-500d338f8c91 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.329517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.330895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.331431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.331887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/812f104fcd164e748c3d6558228d7ef0/roles/22cdd113399a479bb54fcb86fc3f2a15` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.348655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=812f104fcd164e748c3d6558228d7ef0, role_id=22cdd113399a479bb54fcb86fc3f2a15)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:54.350608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:54.403344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:54.428095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f684e08-4d9d-4b3c-a3f9-500d338f8c91 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.439640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 404/799] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:54 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/812f104fcd164e748c3d6558228d7ef0/roles/22cdd113399a479bb54fcb86fc3f2a15 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:54.441066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.477340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3965-95bc-428a-bb17-473d9e8373f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.478157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3965-95bc-428a-bb17-473d9e8373f9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.478440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3965-95bc-428a-bb17-473d9e8373f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.478723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3965-95bc-428a-bb17-473d9e8373f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.490848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3965-95bc-428a-bb17-473d9e8373f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:54.492435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3965-95bc-428a-bb17-473d9e8373f9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-710b3965-95bc-428a-bb17-473d9e8373f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:54.514295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:54.515391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:54.516928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:54.516928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:54.516928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:54.516928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:54.516928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 396/800] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:54.517376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63807ba5-743b-4dd7-b6ae-e7772a0c85ab None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.517674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63807ba5-743b-4dd7-b6ae-e7772a0c85ab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.517960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63807ba5-743b-4dd7-b6ae-e7772a0c85ab None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.519608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 397/801] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:54.521142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63807ba5-743b-4dd7-b6ae-e7772a0c85ab None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.598917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565635b0-ead1-46a3-9579-c6c3ccb82e53 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.600825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:54.640634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2269050-b838-4fc2-be9f-24e157254b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.641426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2269050-b838-4fc2-be9f-24e157254b6f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.641722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2269050-b838-4fc2-be9f-24e157254b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.642003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2269050-b838-4fc2-be9f-24e157254b6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.657430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2269050-b838-4fc2-be9f-24e157254b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:54.664247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63807ba5-743b-4dd7-b6ae-e7772a0c85ab None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f2269050-b838-4fc2-be9f-24e157254b6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:54.666156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:54.667457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 405/802] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:54.668310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4c6f6-1c01-4ed5-8748-a037f9da5aeb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.669090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4c6f6-1c01-4ed5-8748-a037f9da5aeb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.669090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4c6f6-1c01-4ed5-8748-a037f9da5aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.688974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-97e4c6f6-1c01-4ed5-8748-a037f9da5aeb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:54.690267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-97e4c6f6-1c01-4ed5-8748-a037f9da5aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:54.708150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.709103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.709792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.710084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.710818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:54.711129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:54.713085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:54.714922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:54.730699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 398/803] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:54.731559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.776749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 406/804] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:28:54.777652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4c6f6-1c01-4ed5-8748-a037f9da5aeb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.862281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815b6d2-bae7-45d9-8c90-55005beba54d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.891004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:54.898655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.899375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.901759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.901759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.901759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:54.901759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:54.902313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4c6f6-1c01-4ed5-8748-a037f9da5aeb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:54.902914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:54.917007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 399/805] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:54.917007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:54.951783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:54.953608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:54.953962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:54.954308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:54.970643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:54.974040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:54.999126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:55.016112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 407/806] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:55.018374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.023977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:55.091504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d35dc-a96e-46c6-9318-42bb7dc57929 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.131652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.132460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.132773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.133053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.133760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.135665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:55.148888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:55.150573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 17:28:55.151867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:55.153859 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3df9-fb51-4c5e-a47a-4f6807d2fd42 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.155479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 400/807] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Jul 15 17:28:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:55.156572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.196762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.197708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.197992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.198266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.210122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.211553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:55.231936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:55.233360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:55.234879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:55.234879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:55.234879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:55.234879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:55.237034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 408/808] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:55.237914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.243046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:55.320535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e641932-8879-467d-b3c2-fdc10f650bdf None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.344345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:55.362008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.362867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.363152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.363425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.364566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:55.365045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.366682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.387858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 401/809] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:55.388938 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.417800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bd6b5-eae5-4e41-9d07-ec5a5688c261 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.473206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.474968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.475869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.476435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.477365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:55.477837 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.479366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.485134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 409/810] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:55.486087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.553820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd49d9-a9c3-452b-b445-762ed5337f32 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.573240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:55.597168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.597945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.598228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.598505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.599280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:55.599523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.601005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.619604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a9b26-d4cf-4d1a-ad76-8dd7276052c9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.624557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 402/811] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:55.626408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.675116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.676235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.676651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.677041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.692615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.693200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:55.694881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:55.719521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:55.720479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:55.721377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:55.721377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:55.721377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:55.721377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:55.722021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 410/812] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:55.723522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.758725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e492f6-8ca9-48d9-9874-eb422ed0f7fa None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.810923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.812096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.812636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.813264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.814796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:55.815097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.817357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.831020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 403/813] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:55.831020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.855235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0805f-ac94-460a-bdfe-2f7173b205a0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.881834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:55.898913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.899918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.900791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.901262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.910400 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.913042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:55.929932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:55.930916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:28:55.931844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:55.938690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 411/814] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:55.939864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:55.942924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6f2d8-d2fc-4cc1-be2c-1cf05340c107 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:55.945224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:55.980079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:55.980704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:55.980992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:55.981268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:55.981987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:55.982293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:55.984779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:55.996652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 404/815] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:55.997321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:56.050302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98348aa0-f30a-4692-b0fb-ed26be78ebf0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:56.094674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:56.095345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.095628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.095905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.096675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:56.098039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:56.121392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced07cd-8410-46d5-ac3f-e882b5e09f05 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:56.156135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fb749-eeeb-4a56-86ed-2d9f73f010dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:56.156842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fb749-eeeb-4a56-86ed-2d9f73f010dd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.157297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fb749-eeeb-4a56-86ed-2d9f73f010dd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.157453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fb749-eeeb-4a56-86ed-2d9f73f010dd None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.158297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fb749-eeeb-4a56-86ed-2d9f73f010dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:56.158609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fb749-eeeb-4a56-86ed-2d9f73f010dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:56.160052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fb749-eeeb-4a56-86ed-2d9f73f010dd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:56.178723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 405/816] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:28:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:56.180071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baac52-819c-4dca-917a-84436bbe0aaf None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.180361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baac52-819c-4dca-917a-84436bbe0aaf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.180661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baac52-819c-4dca-917a-84436bbe0aaf None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.182230 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 406/817] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:56.184041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baac52-819c-4dca-917a-84436bbe0aaf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:56.205110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 412/818] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:28:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:56.209135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:56.210439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:56.284866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:56.344471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baac52-819c-4dca-917a-84436bbe0aaf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:56.390372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02075bb-f077-48f5-a514-fd9484f4d860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:56.390372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02075bb-f077-48f5-a514-fd9484f4d860 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.390372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02075bb-f077-48f5-a514-fd9484f4d860 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.390981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02075bb-f077-48f5-a514-fd9484f4d860 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.399293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02075bb-f077-48f5-a514-fd9484f4d860 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:56.401414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02075bb-f077-48f5-a514-fd9484f4d860 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-a02075bb-f077-48f5-a514-fd9484f4d860 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:56.417110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:56.418354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:56.419473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:56.422055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 407/819] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:56.423244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b95db-5d91-4c31-aa74-ced2863b571b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.423605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b95db-5d91-4c31-aa74-ced2863b571b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.423984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b95db-5d91-4c31-aa74-ced2863b571b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.443052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-280b95db-5d91-4c31-aa74-ced2863b571b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:56.444424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-280b95db-5d91-4c31-aa74-ced2863b571b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:56.501315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc4091-1351-4df4-bcff-62964a9ee52a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:56.522518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 408/820] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:28:56.523371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b95db-5d91-4c31-aa74-ced2863b571b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:56.532241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:56.551870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d906f03-9449-4dd0-91c0-6ff38e0ab194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:56.552828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d906f03-9449-4dd0-91c0-6ff38e0ab194 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.553263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d906f03-9449-4dd0-91c0-6ff38e0ab194 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.553714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d906f03-9449-4dd0-91c0-6ff38e0ab194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.628144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d906f03-9449-4dd0-91c0-6ff38e0ab194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:56.630070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d906f03-9449-4dd0-91c0-6ff38e0ab194 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:56.633382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b95db-5d91-4c31-aa74-ced2863b571b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:56.668142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:56.669083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:56.669818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.670177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.670529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.671457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:56.672347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:56.673890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:56.686388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 409/821] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:56.687935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:56.738096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 413/822] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:28:56.739597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a059e0-ba39-42c1-8f12-949672e39ff2 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.739992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a059e0-ba39-42c1-8f12-949672e39ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.740376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a059e0-ba39-42c1-8f12-949672e39ff2 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.744752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 414/823] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:56.744752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a059e0-ba39-42c1-8f12-949672e39ff2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:56.773514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:56.806570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81715b8c-6745-475d-942f-d620f62c3dac None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:56.842067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:56.842983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.843867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.844237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] PATH_INFO: `/v3/services/network` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.845000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:56.846479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:56.854927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a059e0-ba39-42c1-8f12-949672e39ff2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:56.859919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:56.861179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 17:28:56.862228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:56.868035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 410/824] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:56 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:56.868629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:56.892892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e63b5-717f-4ee2-a7fe-5ab3d22bdd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:56.893766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e63b5-717f-4ee2-a7fe-5ab3d22bdd58 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.894047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e63b5-717f-4ee2-a7fe-5ab3d22bdd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.894318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e63b5-717f-4ee2-a7fe-5ab3d22bdd58 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/7a49603273fe4eff892e8b024a473158/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.896257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e63b5-717f-4ee2-a7fe-5ab3d22bdd58 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=7a49603273fe4eff892e8b024a473158, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:56.897717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e63b5-717f-4ee2-a7fe-5ab3d22bdd58 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:56.901869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:56.972853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 415/825] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:28:56 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/7a49603273fe4eff892e8b024a473158/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:28:56.974413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51098c7-0014-4bf2-a0d2-dd79fc612250 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:56.974715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51098c7-0014-4bf2-a0d2-dd79fc612250 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:56.974996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51098c7-0014-4bf2-a0d2-dd79fc612250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:56.976052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a1c95-cc16-4625-bfa5-7318ca86e52b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.014480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:57.019368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a51098c7-0014-4bf2-a0d2-dd79fc612250 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:57.021903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.022349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a51098c7-0014-4bf2-a0d2-dd79fc612250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:57.022917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.023216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.023442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.038824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.041030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:57.064179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:57.065593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:57.066506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:57.066506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:57.066506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:57.066506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:57.069702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 411/826] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:57.071375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.126562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 416/827] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:28:57.126562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51098c7-0014-4bf2-a0d2-dd79fc612250 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.210416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb80a8-cdd2-4e8e-bb54-05fad2e1982e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.254698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51098c7-0014-4bf2-a0d2-dd79fc612250 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.258869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.259748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.260198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.260647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.261511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] RBAC: Adding query filter params (name=network) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:57.261986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.263532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.268916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 412/828] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:57.269965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.303085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:57.308776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.309530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.309894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.310248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.312702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:57.317675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.319196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.325387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:57.339764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 417/829] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:57.339764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.396134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866efd37-0dd4-4f50-872b-c080a558c506 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.448429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.449592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.450535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.451007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.463296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:57.466418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72bcf-9827-4762-af9a-53e67357dd07 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:57.471857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:57.473016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:28:57.474114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:57.475476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 413/830] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:57.476411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.513361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.515114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.515549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.515969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.516868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] RBAC: Adding query filter params (type=network) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:57.517339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.518833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.530325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 418/831] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:57.530325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.589889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:57.613408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405685d-cf56-4dc9-954f-5a30c884db55 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.621001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:57.645257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0e89-b280-423b-a162-a97c6f0c0e90 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.650683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.652164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.652649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.653099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.661194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.662896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:57.678781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:57.679885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:57.680947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:57.683247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 414/832] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:57.684306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.695358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.696052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.696858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.696858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.697482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:57.697793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.699578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.712978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 419/833] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:57.713769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.793021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917f95a-2bfb-4b5f-8446-ea86648d8cdc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.825573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276df1a7-2a5d-42e7-8211-30a993a15164 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:57.829220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.830049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.830477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.830892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.831802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.833352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.840421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:57.862657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:57.863329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:57.863614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:57.863895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:57.864731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:57.865044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:57.866477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:57.884334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 420/834] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:57.884334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.932896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 415/835] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:28:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:28:57.934436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:57.983374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:58.077040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de05012-e84f-4ede-9c8c-6bbf0ea24ad3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:58.093015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbed26-f2ac-432f-ade2-5793cc3af928 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:58.153230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2484f1a-3ffd-410d-80ed-8a4b74036665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:58.154034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2484f1a-3ffd-410d-80ed-8a4b74036665 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.154321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2484f1a-3ffd-410d-80ed-8a4b74036665 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.154597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2484f1a-3ffd-410d-80ed-8a4b74036665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.160950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:58.161861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.162344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.162813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.164860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=3ba11eb5837a4567bd77f3fcb0017ea7) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:58.165812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:58.168057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:58.168235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2484f1a-3ffd-410d-80ed-8a4b74036665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:58.169717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2484f1a-3ffd-410d-80ed-8a4b74036665 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:58.181149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 416/836] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/role_assignments?user.id=3ba11eb5837a4567bd77f3fcb0017ea7&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-e2484f1a-3ffd-410d-80ed-8a4b74036665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:58.192781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:58.193717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:58.195045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:58.195045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:58.195045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:58.195045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:58.195045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 421/837] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:58.200335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:58.434817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:58.454336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587ed99-9b6c-4f4d-bac3-58bac26bcd13 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:58.460664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567afc-9699-469d-aa39-429e57775d4c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.460962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567afc-9699-469d-aa39-429e57775d4c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.464669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567afc-9699-469d-aa39-429e57775d4c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.464669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 422/838] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:28:58.468858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5464a2-856b-4ae3-aebd-139a173cacae None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.469151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5464a2-856b-4ae3-aebd-139a173cacae None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.469432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5464a2-856b-4ae3-aebd-139a173cacae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.488057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8f5464a2-856b-4ae3-aebd-139a173cacae None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:58.490103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8f5464a2-856b-4ae3-aebd-139a173cacae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:58.491294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:58.491971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.492261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.492538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.493437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:58.493772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:58.495308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:58.507717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 417/839] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:58.513961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:58.518975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:58.592719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 423/840] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:28:58.609721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5464a2-856b-4ae3-aebd-139a173cacae None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:58.647047 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b1428-4d32-48b0-b487-d49970d25174 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:58.688646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:58.689476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.689854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.690201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.698899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:58.700473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:28:58.718919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:58.720029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:58.721187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 418/841] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:28:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:58.727995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:58.728928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5464a2-856b-4ae3-aebd-139a173cacae None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:58.731674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:58.785317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:58.786409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.786846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.787271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.788201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:58.791006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:58.804675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:58.805864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:28:58.806966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:58.809418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 424/842] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:58.816375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:58.850833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e10fe-31c8-4b1e-9ba8-5989d6e07a49 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:58.851279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:58.903696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:58.904572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:58.905238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:58.905674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:58.907364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:58.907788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:58.910092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:58.936652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 419/843] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 15 17:28:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:58.940966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:58.957130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4471-ac7d-4a91-853c-9c9642352c2f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.010966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.012407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.013213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.013960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.014996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:59.016044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.017754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.028760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 425/844] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:59.030733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.037343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.083041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640f5ae-0890-40f5-ae6c-f25706794ca3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.119346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.120005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.121263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.121263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.129263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.149744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506af5-b263-40d0-afd8-2368e4f900ed None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.185856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabbad4-2b6e-4e8d-b523-cddd4371dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.187609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabbad4-2b6e-4e8d-b523-cddd4371dad2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.188104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabbad4-2b6e-4e8d-b523-cddd4371dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.188582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabbad4-2b6e-4e8d-b523-cddd4371dad2 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.189596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabbad4-2b6e-4e8d-b523-cddd4371dad2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:59.190104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabbad4-2b6e-4e8d-b523-cddd4371dad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.191734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabbad4-2b6e-4e8d-b523-cddd4371dad2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.195766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.197354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 426/845] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:28:59.197583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.199267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608c56-54cc-4587-b2af-d2f9c799da77 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.199765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608c56-54cc-4587-b2af-d2f9c799da77 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.200242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608c56-54cc-4587-b2af-d2f9c799da77 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.201884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 427/846] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:28:59.210101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe6e4d-7b5c-4d44-95c8-5c0820494057 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.210101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe6e4d-7b5c-4d44-95c8-5c0820494057 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.210101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe6e4d-7b5c-4d44-95c8-5c0820494057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.226537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-adbe6e4d-7b5c-4d44-95c8-5c0820494057 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:28:59.227947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-adbe6e4d-7b5c-4d44-95c8-5c0820494057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:28:59.279439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.309038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 428/847] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:28:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:28:59.327579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe6e4d-7b5c-4d44-95c8-5c0820494057 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.329075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 420/848] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:28:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 17:28:59.334993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.423223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.448503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe6e4d-7b5c-4d44-95c8-5c0820494057 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.460391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e254e-0c66-4ce3-a625-536fe69a511f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.485818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.486603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.486887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.487160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.497950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:28:59.504541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.505450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.505801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.506167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:28:59.506274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:59.507124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:59.507961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 429/849] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:59.515334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.527444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.529115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:59.556154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:59.557160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:59.558035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:59.558035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:59.558035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:59.558035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:59.558035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 421/850] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:28:59.563342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.589022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.649008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf3de8-9781-42cf-9d05-03c96a88e8f0 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.680978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ebe19-08e8-498c-937b-c570fcf52cfc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.689205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.690678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.691119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.691546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.692842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:59.693318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.695482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.698819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.708061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 430/851] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:59.713586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.729501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.731202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.732081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.732513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.734569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:28:59.735927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.737562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.760356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 422/852] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:28:59.762130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.827515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6707cd-dbf2-455c-8ba6-3d94c73925f5 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.858316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.865172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.866114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.866538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.866912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.879107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.880682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.898277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccde90-345a-455f-9a03-c131d609ad24 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:28:59.902171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:59.903213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:59.904297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:28:59.904297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:28:59.904297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:28:59.904297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:28:59.910548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 431/853] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:59.911920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:28:59.948639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:28:59.949741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:28:59.950422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:28:59.950863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:28:59.960456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:28:59.962630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:28:59.969944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:28:59.983356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:28:59.995140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 423/854] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:28:59.995997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.030313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9b3b3-6f01-49f6-b5ee-024de1c225fa None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:00.082057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:00.082815 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.083282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.083649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.084887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:00.085009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:00.086608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:00.104093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 432/855] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:00.105680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.110173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd69268-cbeb-479f-8432-41ee04814385 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:00.142170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:00.147641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:00.148561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.149128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.149627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.150891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:00.151463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:00.153101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:00.170931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 424/856] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:28:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:00.172884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.218984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408f5d0-0ae6-431f-a159-c7b6c69860b3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:00.230753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:00.258979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:00.260017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.260561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.260966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.271217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:00.273247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:00.289349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:00.290383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 17:29:00.291310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:00.296066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 433/857] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:29:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:00.298163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.308368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce8aed-ad52-4458-9db3-65964adbbbc1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:00.352659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8fef8-a829-497f-9e06-e1dd378e0c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:00.352659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8fef8-a829-497f-9e06-e1dd378e0c01 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.352659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8fef8-a829-497f-9e06-e1dd378e0c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.352659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8fef8-a829-497f-9e06-e1dd378e0c01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.353353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8fef8-a829-497f-9e06-e1dd378e0c01 None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=812f104fcd164e748c3d6558228d7ef0) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:00.353662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8fef8-a829-497f-9e06-e1dd378e0c01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:00.355258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8fef8-a829-497f-9e06-e1dd378e0c01 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:00.369950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 425/858] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:00 2022] GET /identity/v3/role_assignments?user.id=812f104fcd164e748c3d6558228d7ef0&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:00.431193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e66a-dab0-4070-9d28-79e6836bfeb9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:00.475939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:00.476553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.476869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.477154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.478015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:00.478397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:00.479858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:00.500837 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 434/859] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Jul 15 17:29:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:00.502880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abea1e3-aaf6-45a3-ad87-9a82f5390042 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.503173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abea1e3-aaf6-45a3-ad87-9a82f5390042 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.503462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abea1e3-aaf6-45a3-ad87-9a82f5390042 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.505018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 426/860] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:00.506571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.511548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f9617-802c-4512-9a7d-493e1eed982e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.511863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f9617-802c-4512-9a7d-493e1eed982e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.512152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f9617-802c-4512-9a7d-493e1eed982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.533632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8e2f9617-802c-4512-9a7d-493e1eed982e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:00.534831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8e2f9617-802c-4512-9a7d-493e1eed982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:00.541309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:00.592331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:00.610499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 427/861] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:00.630041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829f563-949f-40a7-8ecd-7732e33f0ae8 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:00.633851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f9617-802c-4512-9a7d-493e1eed982e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.688436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:00.689140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.689425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.690059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.760851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f9617-802c-4512-9a7d-493e1eed982e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:00.789230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:00.790969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:00.812970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:00.814148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:00.815181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:00.815671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:00.816848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:00.819387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:00.821572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:00.837577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:00.838570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:00.839403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:00.843095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 428/862] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Jul 15 17:29:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:00.849437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.911785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 435/863] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:00.917625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:00.918371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:00.994336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e57d0b-61ba-4319-9d5a-9cb48a855afd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.038182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691f8e7-97c3-4036-83da-d88735125360 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.041317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.042181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.042617 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.043041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.043909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:01.044365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.046023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.056095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 429/864] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:01.057311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:01.066721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:01.075949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.080554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.080554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.080554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.089071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.090631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:01.110033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:01.111165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:01.112267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:01.112267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:01.112267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:01.112267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:01.115876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 436/865] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:01.117164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:01.166140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482c0df-86f2-43aa-bb02-f5531c6cc1b5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.180367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:01.200911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.201812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.202247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.203242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.204098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:01.204551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.206101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.217010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 430/866] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:01.227872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf418ba0-15e3-418f-81bd-fbb7cbec872c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.268639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dcd30-d312-4e12-be59-6eaaf2d1bfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.269451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dcd30-d312-4e12-be59-6eaaf2d1bfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.269736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dcd30-d312-4e12-be59-6eaaf2d1bfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.270011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dcd30-d312-4e12-be59-6eaaf2d1bfb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.270763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dcd30-d312-4e12-be59-6eaaf2d1bfb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:01.271068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dcd30-d312-4e12-be59-6eaaf2d1bfb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.272651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dcd30-d312-4e12-be59-6eaaf2d1bfb5 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.292073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 437/867] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:01.293462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:01.410155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0264a5-7330-417e-8172-27b066d49dec None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.415410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf236f-453a-4a98-a209-1a95ad1ede2e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.415759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf236f-453a-4a98-a209-1a95ad1ede2e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.416031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf236f-453a-4a98-a209-1a95ad1ede2e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.417828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 438/868] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:01.425563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fbc07-3381-4b2f-8e23-7ef70cc1bae6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.425976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fbc07-3381-4b2f-8e23-7ef70cc1bae6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.426360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fbc07-3381-4b2f-8e23-7ef70cc1bae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.451045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.452084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.453110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.453531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.457938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-733fbc07-3381-4b2f-8e23-7ef70cc1bae6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:01.459317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-733fbc07-3381-4b2f-8e23-7ef70cc1bae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:01.460970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.462791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:01.479717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:01.480746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:01.481640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:01.484262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 431/869] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:01.489812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:01.501331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:01.548701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 439/870] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:01.548872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:01.568113 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fbc07-3381-4b2f-8e23-7ef70cc1bae6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:01.621356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32779879-d58f-48a4-a1cb-5da2e3f230b7 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.669853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.672071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.672639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.673091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.674398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:01.674864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.677408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.694942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 432/871] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:29:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:01.700676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fbc07-3381-4b2f-8e23-7ef70cc1bae6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.700827 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:01.743173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39105f-fce5-4bf9-bb08-f064c178a5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.744252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39105f-fce5-4bf9-bb08-f064c178a5e2 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.744797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39105f-fce5-4bf9-bb08-f064c178a5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.745305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39105f-fce5-4bf9-bb08-f064c178a5e2 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.746263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39105f-fce5-4bf9-bb08-f064c178a5e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.747847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39105f-fce5-4bf9-bb08-f064c178a5e2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.808216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:01.815719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171a63b-17b4-40ae-a6ee-56c7269a0af8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:01.820546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 440/872] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:29:01 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:29:01.868137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f3b0c-f040-4fcf-9049-5ecbb3a1c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:01.868860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f3b0c-f040-4fcf-9049-5ecbb3a1c50d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:01.869241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f3b0c-f040-4fcf-9049-5ecbb3a1c50d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:01.869582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f3b0c-f040-4fcf-9049-5ecbb3a1c50d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:01.870447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f3b0c-f040-4fcf-9049-5ecbb3a1c50d None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=7a49603273fe4eff892e8b024a473158) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:01.870767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f3b0c-f040-4fcf-9049-5ecbb3a1c50d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:01.872268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f3b0c-f040-4fcf-9049-5ecbb3a1c50d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:01.880533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 433/873] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:01 2022] GET /identity/v3/role_assignments?user.id=7a49603273fe4eff892e8b024a473158&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:02.612183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049a835-9321-4ca0-b669-9d4f25778d1e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:02.612713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049a835-9321-4ca0-b669-9d4f25778d1e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:02.612791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049a835-9321-4ca0-b669-9d4f25778d1e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:02.614345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 441/874] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:02.626026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2f82-4a89-4b91-b45e-6ac190a1bfaf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:02.626316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2f82-4a89-4b91-b45e-6ac190a1bfaf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:02.626595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2f82-4a89-4b91-b45e-6ac190a1bfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:02.643498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b57a2f82-4a89-4b91-b45e-6ac190a1bfaf None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:02.644729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b57a2f82-4a89-4b91-b45e-6ac190a1bfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:02.738322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 434/875] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:02.771901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049a835-9321-4ca0-b669-9d4f25778d1e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:02.825458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:02.924231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049a835-9321-4ca0-b669-9d4f25778d1e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:03.001969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:03.002803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.003087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.003365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.028651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:03.037291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:03.038223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:03.039041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:03.041524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 442/876] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 15 17:29:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:03.050998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2f82-4a89-4b91-b45e-6ac190a1bfaf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:03.169057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:03.178220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2f82-4a89-4b91-b45e-6ac190a1bfaf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:03.212957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5f656-8362-415c-b2c5-4ee480cc460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:03.213847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5f656-8362-415c-b2c5-4ee480cc460e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.214314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5f656-8362-415c-b2c5-4ee480cc460e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.214593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5f656-8362-415c-b2c5-4ee480cc460e None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.215362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5f656-8362-415c-b2c5-4ee480cc460e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:03.215671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5f656-8362-415c-b2c5-4ee480cc460e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:03.217120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5f656-8362-415c-b2c5-4ee480cc460e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:03.226799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 435/877] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 15 17:29:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:03.232111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:03.335240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488709d-9e63-4f15-b1df-4b7f0e96b34e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:03.338927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:03.340505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e280b2-0b42-4161-b2f4-5d6012ab5d63 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.340828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e280b2-0b42-4161-b2f4-5d6012ab5d63 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.343235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e280b2-0b42-4161-b2f4-5d6012ab5d63 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.343235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 436/878] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:03.348509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdadac-e73e-4e5a-823f-925866608852 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.348826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdadac-e73e-4e5a-823f-925866608852 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.349110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdadac-e73e-4e5a-823f-925866608852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.375776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-63fdadac-e73e-4e5a-823f-925866608852 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:03.377461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-63fdadac-e73e-4e5a-823f-925866608852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:03.386982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:03.387787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.388093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.388585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.400349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:03.401815 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:03.419129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:03.420047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:03.421382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:03.421382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:03.421382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:03.421382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:03.425225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 443/879] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:03.426103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:03.460794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 437/880] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:03.479841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdadac-e73e-4e5a-823f-925866608852 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:03.532668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e8e6-d3ba-4806-8fb9-ff569feb62e6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:03.566018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:03.568695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:03.569421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.569710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.570040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.570756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:03.571069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:03.572442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:03.588428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdadac-e73e-4e5a-823f-925866608852 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:03.589751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 444/881] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:03.590800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:03.594198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:03.628463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:03.629412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.629942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.630422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.631354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:03.632960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:03.716723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f1640-c0f4-4160-9b3a-f5caccae4591 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:03.717714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 438/882] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:29:03 2022] POST /identity/v3/services => generated 269 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:29:03.761170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:03.762004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.762291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.762568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.772128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:03.773673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:03.792216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:03.793244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:03.794192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 445/883] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:29:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:03.799846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:03.910641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c0685-59a8-4369-bbd0-c7ade436ef29 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:03.945608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4d57d-3660-494b-9182-ecf03406f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:03.946290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4d57d-3660-494b-9182-ecf03406f421 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:03.946576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4d57d-3660-494b-9182-ecf03406f421 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:03.946855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4d57d-3660-494b-9182-ecf03406f421 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:03.947702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4d57d-3660-494b-9182-ecf03406f421 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:03.948017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4d57d-3660-494b-9182-ecf03406f421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:03.949490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4d57d-3660-494b-9182-ecf03406f421 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:03.959207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:03.968954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 439/884] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 15 17:29:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:03.974575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.039716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:04.051199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aa52d-4245-4490-99d4-ec8b3d88ac09 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.051692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aa52d-4245-4490-99d4-ec8b3d88ac09 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.051895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aa52d-4245-4490-99d4-ec8b3d88ac09 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.053855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 440/885] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:04.059971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396bf0-11fb-48c9-93a6-19c6e10c9b7c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.060540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396bf0-11fb-48c9-93a6-19c6e10c9b7c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.061080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396bf0-11fb-48c9-93a6-19c6e10c9b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.079002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-97396bf0-11fb-48c9-93a6-19c6e10c9b7c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:04.080523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-97396bf0-11fb-48c9-93a6-19c6e10c9b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:04.096152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97b061-f1aa-48ed-9bed-98718ab0e91a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.138660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.139304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.139590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.140643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.159650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 441/886] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:04.166170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1666a30-caa3-41eb-a345-08af4121bd5b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.166473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1666a30-caa3-41eb-a345-08af4121bd5b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.166796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1666a30-caa3-41eb-a345-08af4121bd5b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.168295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 442/887] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:04.173538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f3377-6dcd-4e3e-9673-3cc5e3b6e432 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.173833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f3377-6dcd-4e3e-9673-3cc5e3b6e432 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.174120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f3377-6dcd-4e3e-9673-3cc5e3b6e432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.191581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4e5f3377-6dcd-4e3e-9673-3cc5e3b6e432 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:04.192818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4e5f3377-6dcd-4e3e-9673-3cc5e3b6e432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:04.225034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.226461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.268664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 443/888] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:04.269907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f3377-6dcd-4e3e-9673-3cc5e3b6e432 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.312940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:04.340991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:04.361248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 446/889] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:03 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:04.362803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.383411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f3377-6dcd-4e3e-9673-3cc5e3b6e432 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.423935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.424787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.425080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.425360 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.426016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.427395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:04.439695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:04.440771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:04.442251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 444/890] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:04.443155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.484586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47477e33-3fb9-45ad-a457-f53b380a0798 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.521456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.522274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.522539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.522812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.523458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.524842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:04.536898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:04.537921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:04.538759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 447/891] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:04.540265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.554322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbdeab-ced6-462a-9802-cac4bbbc36c3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.566668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:04.577135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:04.588329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.589153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.589443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.589725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.601031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.602491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:04.620383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:04.621464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:04.622390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:04.622390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:04.622390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:04.622390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:04.622390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 445/892] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:04.622390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.649874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63c58c-ddb2-4f45-b6d4-63f76c7b3ced None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.686620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.687276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.687574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.687830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.688530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:04.688864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.690216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.699536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 448/893] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:04.699536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.743029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0befc-0468-4784-afd1-64d6f22d0018 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.782291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.783433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.783721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.784394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.785416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:04.785731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.787378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.791553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:04.799236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 446/894] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:04.799236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.816970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fafc9c-585e-4b09-9aed-1459a8f1acd7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.842639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.843869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.844154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.844429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.845286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:04.845715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:04.845848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.847223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.864043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 449/895] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:04.864043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.915852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e2b7a-1ced-488d-a3cf-d8b33fb2c3e2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:04.953677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:04.954379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:04.954664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:04.954950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:04.955664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:04.955974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:04.957452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:04.964744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 447/896] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:04.965698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:04.984999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dbaea-55cb-4274-b03d-9344b1d7ad74 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:05.036666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:05.038443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.039223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.039701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.040837 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:05.041421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:05.044093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:05.055993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 450/897] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:05.062603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:05.079775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:05.085055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c66e44-2af4-489f-ad29-93e81ace93e9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:05.087737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:05.130902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:05.132036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.132495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.132808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.140953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:05.143750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:05.162924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:05.164051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:05.165200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 448/898] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:05.167529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:05.217009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327803-4be4-475b-b508-c05681d7ccca None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:05.265663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:05.267209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.267684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.268157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.269188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] RBAC: Adding query filter params (interface=public, service_id=ecdf71e8a6ea421883bcdc25929b268b, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:05.269757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:05.272134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:05.283775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 451/899] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:29:05 2022] GET /identity/v3/endpoints?service_id=ecdf71e8a6ea421883bcdc25929b268b&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:05.284521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23fa8d-7f06-49ae-8599-3126ac1c31e6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:05.288990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:05.329170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:05.329844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.331031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.331334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.332149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:05.332472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:05.333989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:05.355847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 449/900] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:05.357385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:05.382296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:05.385976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:05.411574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793bb75-0ed4-4dee-ac7e-15633028e381 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:05.448076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116caff-b5b3-423e-81a2-7fcfd6a158cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:05.449114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116caff-b5b3-423e-81a2-7fcfd6a158cb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.449616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116caff-b5b3-423e-81a2-7fcfd6a158cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.449994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116caff-b5b3-423e-81a2-7fcfd6a158cb None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.450786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116caff-b5b3-423e-81a2-7fcfd6a158cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:05.452807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116caff-b5b3-423e-81a2-7fcfd6a158cb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:05.459156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 452/901] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:29:05 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:29:05.479400 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf43e6-a0c2-4871-997b-161eceda1008 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:05.520148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6e3bb-8c45-4d65-9591-68cf1bcf3a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:05.520860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e3bb-8c45-4d65-9591-68cf1bcf3a0f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.521145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e3bb-8c45-4d65-9591-68cf1bcf3a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.521422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e3bb-8c45-4d65-9591-68cf1bcf3a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.522272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6e3bb-8c45-4d65-9591-68cf1bcf3a0f None admin] RBAC: Adding query filter params (role.id=d3f1418897324c3d88c6f01b75cf98e9, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=812f104fcd164e748c3d6558228d7ef0) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:05.522585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6e3bb-8c45-4d65-9591-68cf1bcf3a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:05.524062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6e3bb-8c45-4d65-9591-68cf1bcf3a0f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:05.531819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 450/902] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:05 2022] GET /identity/v3/role_assignments?user.id=812f104fcd164e748c3d6558228d7ef0&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=d3f1418897324c3d88c6f01b75cf98e9 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:05.817105 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336f8f-faca-46ac-822b-c5e2b1b06e55 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.817651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336f8f-faca-46ac-822b-c5e2b1b06e55 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.817935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336f8f-faca-46ac-822b-c5e2b1b06e55 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.819677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 453/903] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:05.825160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf88f3-33a6-43cb-92cf-38486dd56865 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:05.825734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf88f3-33a6-43cb-92cf-38486dd56865 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:05.826440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf88f3-33a6-43cb-92cf-38486dd56865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:05.844144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-19bf88f3-33a6-43cb-92cf-38486dd56865 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:05.845703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19bf88f3-33a6-43cb-92cf-38486dd56865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:05.915468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 451/904] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:05.938096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e336f8f-faca-46ac-822b-c5e2b1b06e55 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:06.053067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e336f8f-faca-46ac-822b-c5e2b1b06e55 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:06.089336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:06.090502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:06.090987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:06.091461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:06.092410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:06.094765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:06.108472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:06.109465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:06.110300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:06.113266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 454/905] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Jul 15 17:29:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:06.121808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf88f3-33a6-43cb-92cf-38486dd56865 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:06.141619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:06.226976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf88f3-33a6-43cb-92cf-38486dd56865 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:06.266100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:06.267062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:06.267584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:06.268086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:06.269042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:06.269566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:06.271140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:06.276554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 452/906] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:06.288645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:06.288645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:06.396112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b7188-1c7c-4a50-9457-cc08505e6434 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:06.431661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:06.432374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:06.432710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:06.432996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:06.433709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:06.434021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:06.435384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:06.441415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 455/907] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:06.447041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:06.533503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:06.553261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca29d-aace-4c40-91a4-df5f481d83b0 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:06.588707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a56cc-9ea5-41b0-9099-2db4e1b5a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:06.589430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a56cc-9ea5-41b0-9099-2db4e1b5a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:06.589723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a56cc-9ea5-41b0-9099-2db4e1b5a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:06.590007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a56cc-9ea5-41b0-9099-2db4e1b5a0d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:06.590813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a56cc-9ea5-41b0-9099-2db4e1b5a0d2 None admin] RBAC: Adding query filter params (interface=public, service_id=800e4639a7ff4d5e85a2daef3c04f844, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:06.591132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a56cc-9ea5-41b0-9099-2db4e1b5a0d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:06.592523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a56cc-9ea5-41b0-9099-2db4e1b5a0d2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:06.598838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 453/908] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:29:06 2022] GET /identity/v3/endpoints?service_id=800e4639a7ff4d5e85a2daef3c04f844&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:06.604494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:06.677070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:06.715169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fd853-d503-49af-a65b-da3e57b7cd9c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:06.754406 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db253378-1059-4b7f-8750-b27142947852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:06.755085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253378-1059-4b7f-8750-b27142947852 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:06.755371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253378-1059-4b7f-8750-b27142947852 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:06.755651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253378-1059-4b7f-8750-b27142947852 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:06.756343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db253378-1059-4b7f-8750-b27142947852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:06.757749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db253378-1059-4b7f-8750-b27142947852 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:06.764176 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 456/909] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:29:06 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:29:07.084923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a6aa8-be12-4550-8ab5-e9fdd1a34257 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.085640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a6aa8-be12-4550-8ab5-e9fdd1a34257 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.086165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a6aa8-be12-4550-8ab5-e9fdd1a34257 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.088145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 454/910] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:07.094730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f436-5491-403a-9482-142b45258b7a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.095706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f436-5491-403a-9482-142b45258b7a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.096195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f436-5491-403a-9482-142b45258b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.115077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7869f436-5491-403a-9482-142b45258b7a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:07.116672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7869f436-5491-403a-9482-142b45258b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:07.186168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 457/911] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:07.209929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a6aa8-be12-4550-8ab5-e9fdd1a34257 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:07.326520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a6aa8-be12-4550-8ab5-e9fdd1a34257 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:07.361416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8282d7-2c2e-40e9-a56e-afa3d9d6c7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:07.362004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8282d7-2c2e-40e9-a56e-afa3d9d6c7eb None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.362292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8282d7-2c2e-40e9-a56e-afa3d9d6c7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.362575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8282d7-2c2e-40e9-a56e-afa3d9d6c7eb None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.363314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8282d7-2c2e-40e9-a56e-afa3d9d6c7eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:07.364730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8282d7-2c2e-40e9-a56e-afa3d9d6c7eb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:07.443408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 455/912] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:29:07 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:07.565701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0475ae-bd41-4d57-8052-f40ca61f3d8e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.566040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0475ae-bd41-4d57-8052-f40ca61f3d8e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.566278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0475ae-bd41-4d57-8052-f40ca61f3d8e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.568011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 458/913] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:07.573752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1fc7-24cd-4b85-8dc7-95e4afa13971 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.574049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1fc7-24cd-4b85-8dc7-95e4afa13971 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.574334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1fc7-24cd-4b85-8dc7-95e4afa13971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.580620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:07.593529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-adec1fc7-24cd-4b85-8dc7-95e4afa13971 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:07.595820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-adec1fc7-24cd-4b85-8dc7-95e4afa13971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:07.675890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 456/914] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:07.698974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0475ae-bd41-4d57-8052-f40ca61f3d8e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:07.709762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304891e5-be8c-4ea5-83a4-d621e44d2b22 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.710314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304891e5-be8c-4ea5-83a4-d621e44d2b22 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.710805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304891e5-be8c-4ea5-83a4-d621e44d2b22 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.712421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 457/915] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:07.717614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc212991-84e0-43d1-a06b-d1b1e03729ba None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.718150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc212991-84e0-43d1-a06b-d1b1e03729ba None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.718616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc212991-84e0-43d1-a06b-d1b1e03729ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.739441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fc212991-84e0-43d1-a06b-d1b1e03729ba None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:07.740987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc212991-84e0-43d1-a06b-d1b1e03729ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:07.781002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:07.823838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 458/916] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:07.833732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0475ae-bd41-4d57-8052-f40ca61f3d8e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:07.847777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc212991-84e0-43d1-a06b-d1b1e03729ba None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:07.878374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:07.879794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:07.880523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:07.881435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:07.882612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:07.884472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:07.901831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:07.902886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 17:29:07.903710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:07.906956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 459/917] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:29:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:07.913073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:07.921222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:07.969679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc212991-84e0-43d1-a06b-d1b1e03729ba None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.004266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.005130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.005508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.005789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.017182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:08.025706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:08.026653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:08.027657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 459/918] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 15 17:29:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:08.032833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:08.047528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbdf09-5ca6-45df-a91a-faca5a602766 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.072417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:08.084430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.092829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.092829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.092829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.092829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:08.092829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.092829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.094009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 460/919] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:29:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:08.100669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:08.152833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c99f0-6007-417f-a594-a9a167b51383 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.173425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:08.188748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.189498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.189787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.190069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.190822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:08.191135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.192686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.204627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 460/920] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 15 17:29:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:08.208370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:08.223643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee1594-e8d3-4e56-a328-bca45b91300c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.262469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.263049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.263353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.263622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.264338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:08.265202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.266627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.276674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 461/921] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Jul 15 17:29:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:08.278607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:08.336962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69e175-f0d0-4a7f-b677-0cc3d0e7f440 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.359261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:08.384860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.385736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.386030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.386311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.398936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.401030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:08.420353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:08.423096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:08.424001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:08.424001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:08.424001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:08.424001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:08.424228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90e489-086d-45eb-8d7f-a7bac4b4bb2b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.426446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 461/922] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:08.434691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:08.466752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.467850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.468362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.468896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.469850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.471430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.472512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:08.548740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 462/923] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:29:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:08.553442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8e258-808a-4599-8237-e43eb39ff7db None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.601975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.602825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.603129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.603415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.604214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:08.604533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.606100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.619993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 462/924] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:08.626559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:08.751461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4b844-1dc9-4c95-a39d-0c01c76943cd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.761723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:08.788027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd046c6-1934-45cd-813a-b820e4f64292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.788873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd046c6-1934-45cd-813a-b820e4f64292 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.789171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd046c6-1934-45cd-813a-b820e4f64292 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.789454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd046c6-1934-45cd-813a-b820e4f64292 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.798537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd046c6-1934-45cd-813a-b820e4f64292 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.800810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd046c6-1934-45cd-813a-b820e4f64292 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8dd046c6-1934-45cd-813a-b820e4f64292 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:08.817663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:08.818677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:08.819635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 463/925] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:29:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:08.826866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:08.864933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dd170-0be4-4d5e-bece-8af252e7767d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.865296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dd170-0be4-4d5e-bece-8af252e7767d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.865505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3dd170-0be4-4d5e-bece-8af252e7767d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.867119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 464/926] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:08.872039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91031f70-c4be-4006-a46c-48c5779b246c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.872581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91031f70-c4be-4006-a46c-48c5779b246c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.873208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91031f70-c4be-4006-a46c-48c5779b246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.892341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-91031f70-c4be-4006-a46c-48c5779b246c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:08.894049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-91031f70-c4be-4006-a46c-48c5779b246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:08.924550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:08.934243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bfb2c-9d1f-407e-a1fe-619bdeeb77f9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:08.969747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:08.970518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:08.970812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:08.971092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:08.971934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:08.972250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:08.973287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 465/927] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:08.973733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:08.997838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91031f70-c4be-4006-a46c-48c5779b246c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.003225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 463/928] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 15 17:29:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:09.009423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.109131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:09.113402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91031f70-c4be-4006-a46c-48c5779b246c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.148671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e8a83-20af-44a1-b302-b0e496bf8966 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.151616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.152643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.152767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.153054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.153714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.155091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:09.168371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:09.169612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:09.170565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 466/929] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Jul 15 17:29:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:09.176584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.194708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.196124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.196908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.197768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.212849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.214893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.226843 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:09.243456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:09.244376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:09.245311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:09.245311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:09.245311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:09.245311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:09.249483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 464/930] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:09.250882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.297218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cc03f-589a-4145-9d5f-648c7d938cab None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.345288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.345960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.346251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.346533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.347255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:09.347570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.349511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.362009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 467/931] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:09.362878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.383456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2510b-310c-45ea-9334-b66aa38609fd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.397892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:09.423436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.424472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.424990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.425489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.426553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:09.427079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.428745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.442105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 465/932] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:09.448480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.509598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3f7e7-02eb-4ab4-a82f-1a1a2fce9dd6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.523491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:09.547289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.548312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.548825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.549321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.550229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:09.550735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.552312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.562927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 468/933] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:09.564075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.573044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be476066-4a8e-4223-92fe-6ae6d06041a1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.614309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864d111-ab18-4544-950e-4e5ed57420c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.615138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d111-ab18-4544-950e-4e5ed57420c6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.615428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d111-ab18-4544-950e-4e5ed57420c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.615715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d111-ab18-4544-950e-4e5ed57420c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.622706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864d111-ab18-4544-950e-4e5ed57420c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.624738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864d111-ab18-4544-950e-4e5ed57420c6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d864d111-ab18-4544-950e-4e5ed57420c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:09.641846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:09.642801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:09.643671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:09.647964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 466/934] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:09.649136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91a341-4048-488d-8cc1-4c36ae78ae0e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.649434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91a341-4048-488d-8cc1-4c36ae78ae0e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.649727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91a341-4048-488d-8cc1-4c36ae78ae0e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.651530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 467/935] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:09.652245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91a341-4048-488d-8cc1-4c36ae78ae0e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.666851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:09.678158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d0c4b-dc25-4c64-a7de-8228aa8fe236 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.717280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.718762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.718882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.719122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.719872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.721259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.761140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91a341-4048-488d-8cc1-4c36ae78ae0e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:09.783763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:09.797767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95862f8-b14d-4d18-b136-a44527eb6aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:09.798439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95862f8-b14d-4d18-b136-a44527eb6aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.798730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95862f8-b14d-4d18-b136-a44527eb6aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.799011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95862f8-b14d-4d18-b136-a44527eb6aa2 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.799846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95862f8-b14d-4d18-b136-a44527eb6aa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:09.800158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95862f8-b14d-4d18-b136-a44527eb6aa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:09.801666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95862f8-b14d-4d18-b136-a44527eb6aa2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:09.813125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 468/936] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:09.814402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de76580-2774-49d9-b6ce-6961bb9de643 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:09.814702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de76580-2774-49d9-b6ce-6961bb9de643 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:09.814986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de76580-2774-49d9-b6ce-6961bb9de643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:09.825921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 469/937] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:29:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:09.834055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8de76580-2774-49d9-b6ce-6961bb9de643 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:09.835388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8de76580-2774-49d9-b6ce-6961bb9de643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:09.837800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.938517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 469/938] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:09.961086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de76580-2774-49d9-b6ce-6961bb9de643 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:09.998795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c2a62-3a97-419f-a46b-78ca688f63df None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:10.038412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:10.039304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.039582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.039946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/812f104fcd164e748c3d6558228d7ef0/roles/d3f1418897324c3d88c6f01b75cf98e9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.043237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=812f104fcd164e748c3d6558228d7ef0, role_id=d3f1418897324c3d88c6f01b75cf98e9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:10.044718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:10.089117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de76580-2774-49d9-b6ce-6961bb9de643 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:10.097191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:10.129384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 470/939] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:29:09 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/812f104fcd164e748c3d6558228d7ef0/roles/d3f1418897324c3d88c6f01b75cf98e9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:29:10.129963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:10.129963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.129963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.129963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.130246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:10.130246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:10.144514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:10.149272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:10.150388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 470/940] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:10.155712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:10.169001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:10.273070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6e965-0291-46c1-873b-5fe5ccd16ef4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:10.312073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:10.312787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.313077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.313416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.314137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:10.314521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:10.315886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:10.321457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 471/941] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:10.328096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:10.450932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0a14-39b6-4f54-b907-2fef2675b91e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:10.495952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:10.495952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.495952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.495952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.496393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:10.496393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:10.496393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:10.501990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 471/942] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:10.508384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:10.581549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:10.618192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7e292-f472-4f32-bf9b-2652fb27908d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:10.660773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473046e4-a27b-4174-8918-fa1e29a106f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:10.661787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473046e4-a27b-4174-8918-fa1e29a106f6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.662339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473046e4-a27b-4174-8918-fa1e29a106f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.662819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473046e4-a27b-4174-8918-fa1e29a106f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.663860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473046e4-a27b-4174-8918-fa1e29a106f6 None admin] RBAC: Adding query filter params (interface=public, service_id=daa612f2ffa7420095c333a678198ab9, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:10.664369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473046e4-a27b-4174-8918-fa1e29a106f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:10.666458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473046e4-a27b-4174-8918-fa1e29a106f6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:10.673311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 472/943] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:29:10 2022] GET /identity/v3/endpoints?service_id=daa612f2ffa7420095c333a678198ab9&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:10.679009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:10.702993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781560-fb64-482d-9a3b-8fc756417826 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.703705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781560-fb64-482d-9a3b-8fc756417826 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.704233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781560-fb64-482d-9a3b-8fc756417826 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.706095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 473/944] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:10.711767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b6d6-7ec8-4e77-854c-0db03bef8934 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.712081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b6d6-7ec8-4e77-854c-0db03bef8934 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.712357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b6d6-7ec8-4e77-854c-0db03bef8934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.722803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:10.729088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ddb1b6d6-7ec8-4e77-854c-0db03bef8934 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:10.730279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ddb1b6d6-7ec8-4e77-854c-0db03bef8934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:10.797803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70768a-243f-4d33-9786-60c4f65b6f68 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:10.823975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 474/945] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:10.837466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:10.838329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:10.838821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:10.839241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:10.840118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:10.841704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:10.847606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b6d6-7ec8-4e77-854c-0db03bef8934 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:10.860949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 472/946] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:29:10 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:29:10.918565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:10.960404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b6d6-7ec8-4e77-854c-0db03bef8934 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:11.000153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:11.000863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:11.001153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:11.001437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:11.002177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:11.003640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:11.005075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING keystone.server.flask.common [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 17:29:11.058734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.sql.core [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 17:29:11.058734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 17:29:11.058734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [parameters: {'id': '815f6e3b41074a9a9a59dac5a7f02c3c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 17:29:11.058734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80471) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 815f6e3b41074a9a9a59dac5a7f02c3c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 815f6e3b41074a9a9a59dac5a7f02c3c. Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application conn.query(q) Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 17:29:11.076322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result.read() Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 15 17:29:11.077253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application next(self.gen) Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application yield resource Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application next(self.gen) Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise self.value Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application session.commit() Jul 15 17:29:11.078184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.session.flush() Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application compat.raise_( Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 17:29:11.079072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 17:29:11.079976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application conn.query(q) Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application result.read() Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 17:29:11.080795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [parameters: {'id': '815f6e3b41074a9a9a59dac5a7f02c3c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:11.081672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:11.082440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 815f6e3b41074a9a9a59dac5a7f02c3c. Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:11.083354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 475/947] 10.222.0.57 () {60 vars in 1140 bytes} [Fri Jul 15 17:29:10 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:11.083888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:11.192960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:11.197440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adefbc6-6169-4e1f-b632-1fdda96ca84e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:11.235172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac5d0f-ad52-406f-9675-3fd6892ac46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:11.236131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac5d0f-ad52-406f-9675-3fd6892ac46f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:11.236477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac5d0f-ad52-406f-9675-3fd6892ac46f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:11.236861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac5d0f-ad52-406f-9675-3fd6892ac46f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:11.243776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac5d0f-ad52-406f-9675-3fd6892ac46f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:11.245461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac5d0f-ad52-406f-9675-3fd6892ac46f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-7dac5d0f-ad52-406f-9675-3fd6892ac46f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:11.259790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:11.260936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:11.261882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:11.262310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 473/948] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:11.268896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:11.378943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c04b96-9562-4210-872c-c9bb8e3e7dc7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:11.401914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:11.417524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375ab21-72ee-4ac3-9a36-d4d3a25d7c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:11.418193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ab21-72ee-4ac3-9a36-d4d3a25d7c5a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:11.418483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ab21-72ee-4ac3-9a36-d4d3a25d7c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:11.418768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ab21-72ee-4ac3-9a36-d4d3a25d7c5a None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:11.419544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375ab21-72ee-4ac3-9a36-d4d3a25d7c5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:11.419858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375ab21-72ee-4ac3-9a36-d4d3a25d7c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:11.422219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375ab21-72ee-4ac3-9a36-d4d3a25d7c5a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:11.443795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 476/949] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:29:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:11.975295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664135bd-3b8a-457f-802c-44e2d5eb7f5e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:11.975842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664135bd-3b8a-457f-802c-44e2d5eb7f5e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:11.976137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664135bd-3b8a-457f-802c-44e2d5eb7f5e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:11.977936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 474/950] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:11.983973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded596b-e481-4d36-8bd1-e3a5eaa61225 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:11.984472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded596b-e481-4d36-8bd1-e3a5eaa61225 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:11.984961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded596b-e481-4d36-8bd1-e3a5eaa61225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.004401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dded596b-e481-4d36-8bd1-e3a5eaa61225 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:12.006111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dded596b-e481-4d36-8bd1-e3a5eaa61225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:12.085203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 477/951] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:12.104289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664135bd-3b8a-457f-802c-44e2d5eb7f5e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:12.211771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664135bd-3b8a-457f-802c-44e2d5eb7f5e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:12.257401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58a16f-93a0-410e-ac2d-dcd374648e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:12.258204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a16f-93a0-410e-ac2d-dcd374648e4e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.258516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a16f-93a0-410e-ac2d-dcd374648e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.258844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a16f-93a0-410e-ac2d-dcd374648e4e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.259429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58a16f-93a0-410e-ac2d-dcd374648e4e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:12.260854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58a16f-93a0-410e-ac2d-dcd374648e4e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1e58a16f-93a0-410e-ac2d-dcd374648e4e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:12.273124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:12.274210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:12.275522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:12.279519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 475/952] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:12.281478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded596b-e481-4d36-8bd1-e3a5eaa61225 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:12.326918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fca449-d406-4063-9525-059b469748ee None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.327737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fca449-d406-4063-9525-059b469748ee None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.328002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fca449-d406-4063-9525-059b469748ee None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.329680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 476/953] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:12.334487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991412c-5293-446b-8e2a-084cbb9d398d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.334786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991412c-5293-446b-8e2a-084cbb9d398d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.335079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991412c-5293-446b-8e2a-084cbb9d398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.352263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d991412c-5293-446b-8e2a-084cbb9d398d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:12.358957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d991412c-5293-446b-8e2a-084cbb9d398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:12.392967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:12.394154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded596b-e481-4d36-8bd1-e3a5eaa61225 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:12.406251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:12.431196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 477/954] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:12.431683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:12.432502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.432836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.433122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.433841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:12.434153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:12.435528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:12.441213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 478/955] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:29:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:12.449930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991412c-5293-446b-8e2a-084cbb9d398d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:12.461936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:12.567379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991412c-5293-446b-8e2a-084cbb9d398d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:12.578329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a4af8-7636-437c-9fde-550587795f1f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:12.605515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:12.606203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.606490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.606849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.607597 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:12.607912 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:12.610069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:12.615474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 478/956] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:29:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:12.624096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:12.625388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.625899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.626419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.643938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:12.653115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:12.654119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:12.655021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:12.658801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 479/957] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 15 17:29:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:12.665611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:12.693064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:12.781422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5441af4-0374-47e4-884d-78a62428578e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:12.816919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:12.817643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.817870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.818147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.818931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:12.819250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:12.820701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:12.830183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 479/958] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 15 17:29:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:12.835732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:12.873160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:12.943277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f776bb5-58be-43f1-a3aa-3d4c88cd7fcd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:12.979996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37450abc-fc16-429a-af90-13d313d06f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:12.980876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450abc-fc16-429a-af90-13d313d06f92 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:12.981299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450abc-fc16-429a-af90-13d313d06f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:12.981577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450abc-fc16-429a-af90-13d313d06f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:12.993475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37450abc-fc16-429a-af90-13d313d06f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:12.994980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37450abc-fc16-429a-af90-13d313d06f92 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-37450abc-fc16-429a-af90-13d313d06f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:13.013263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:13.014192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:13.015107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:13.015107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:13.015107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:13.015107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:13.015107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 480/959] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:13.020395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.030203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082900c6-117f-470b-8acb-5a1889268a28 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.030501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082900c6-117f-470b-8acb-5a1889268a28 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.030788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082900c6-117f-470b-8acb-5a1889268a28 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.032419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 481/960] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:13.037391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6036ab-602d-4a2c-9ff9-b0b208f16d87 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.037685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6036ab-602d-4a2c-9ff9-b0b208f16d87 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.037968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6036ab-602d-4a2c-9ff9-b0b208f16d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.058234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-be6036ab-602d-4a2c-9ff9-b0b208f16d87 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:13.059967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-be6036ab-602d-4a2c-9ff9-b0b208f16d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:13.117219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:13.148649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 482/961] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:13.151739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2f198-abb6-4f3f-856a-35bd41fc763d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.165889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6036ab-602d-4a2c-9ff9-b0b208f16d87 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.187857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:13.199368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.200799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.201266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.203376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.203376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:13.203376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.205657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.222649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 480/962] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:13.237404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.305285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6036ab-602d-4a2c-9ff9-b0b208f16d87 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.353904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.355634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.356408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.356919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.358117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.360447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.369984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7129b56-9c2e-4867-acc0-ff38e024cc84 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:13.377778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:13.378761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:13.379624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 483/963] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:13.385409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.414010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.415836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.416273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.417042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.427692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.429429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.432901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:13.450767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:13.451761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 17:29:13.452790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:13.455482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 481/964] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:29:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:13.461040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.489501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:13.511322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cace04-496b-40d9-99ba-dc6eb3261395 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.548108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.548929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.549281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.549636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.550412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:13.550858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.552324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.557833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 484/965] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:13.563701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.584389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207b747-1a2b-46f1-a196-225bb8d25434 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.620567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e19df-4a58-472b-93f0-6c290d12c6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.621414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e19df-4a58-472b-93f0-6c290d12c6af None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.621767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e19df-4a58-472b-93f0-6c290d12c6af None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.622138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e19df-4a58-472b-93f0-6c290d12c6af None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.623108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e19df-4a58-472b-93f0-6c290d12c6af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:13.623491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e19df-4a58-472b-93f0-6c290d12c6af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.625127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e19df-4a58-472b-93f0-6c290d12c6af None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.648728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 482/966] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Jul 15 17:29:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:13.650939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a8ff9-3585-4c16-b7d5-ecb105cb1a90 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.651233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a8ff9-3585-4c16-b7d5-ecb105cb1a90 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.651511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a8ff9-3585-4c16-b7d5-ecb105cb1a90 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.652979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 483/967] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:13.653560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a8ff9-3585-4c16-b7d5-ecb105cb1a90 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.681602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d8355-3a98-4f88-abff-5e6b32119801 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.715236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:13.720981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfd9bc-5a72-44bd-aaf8-363905b138b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.721654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfd9bc-5a72-44bd-aaf8-363905b138b6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.721942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfd9bc-5a72-44bd-aaf8-363905b138b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.725505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfd9bc-5a72-44bd-aaf8-363905b138b6 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.725505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfd9bc-5a72-44bd-aaf8-363905b138b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:13.725505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfd9bc-5a72-44bd-aaf8-363905b138b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.725505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfd9bc-5a72-44bd-aaf8-363905b138b6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.734994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 485/968] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:13.734994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c048ff-2295-4bc2-aa45-51524af6dfb6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.734994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c048ff-2295-4bc2-aa45-51524af6dfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.734994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c048ff-2295-4bc2-aa45-51524af6dfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.752228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e1c048ff-2295-4bc2-aa45-51524af6dfb6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:13.753462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e1c048ff-2295-4bc2-aa45-51524af6dfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:13.755682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:13.768293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a8ff9-3585-4c16-b7d5-ecb105cb1a90 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.806993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.807752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.808105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.808453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.840626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 486/969] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:13.840626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c048ff-2295-4bc2-aa45-51524af6dfb6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.879903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.881510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:13.939935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:13.950079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c048ff-2295-4bc2-aa45-51524af6dfb6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:13.985802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:13.986465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:13.986756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:13.987222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 484/970] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:13.987932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:13.991782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:13.992554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:13.993974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.001988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:14.064636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 487/971] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:29:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:14.065743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.110166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5408ca-446a-432e-b8ae-dbbd93c1df65 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:14.157916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:14.158608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.158957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.159322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.160094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:14.162466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.164269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING keystone.server.flask.common [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 17:29:14.203731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb54ab-e1bd-44d2-8fa2-74bda159643f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:14.208106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.sql.core [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 17:29:14.208106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 17:29:14.208106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [parameters: {'user_id': '64b748fc21f34b3c80a69169abd11499', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 17:29:14.208106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80472) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 15 17:29:14.219317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 17:29:14.220271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application yield resource Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application next(self.gen) Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise self.value Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application session.commit() Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 17:29:14.221304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.session.flush() Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application compat.raise_( Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 17:29:14.222255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 17:29:14.223149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application conn.query(q) Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application result.read() Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 17:29:14.228560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [parameters: {'user_id': '64b748fc21f34b3c80a69169abd11499', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:14.229448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 15 17:29:14.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 15 17:29:14.231145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 15 17:29:14.231145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 15 17:29:14.231145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 17:29:14.231145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 17:29:14.231145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 17:29:14.231145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:14.231145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 485/972] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Jul 15 17:29:13 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:14.231556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.252679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:14.271057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:14.271866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.275393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.275393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.285317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:14.286769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:14.308920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:14.310714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:14.311761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:14.311761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:14.311761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:14.311761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:14.311761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 488/973] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:14.321715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.366003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6b8c2-5342-4904-a6d3-3e68e447b423 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:14.383653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:14.415176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618723f8-3725-4499-becc-7645210577b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:14.416003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618723f8-3725-4499-becc-7645210577b9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.416361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618723f8-3725-4499-becc-7645210577b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.416751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618723f8-3725-4499-becc-7645210577b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.424581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618723f8-3725-4499-becc-7645210577b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:14.428000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618723f8-3725-4499-becc-7645210577b9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-618723f8-3725-4499-becc-7645210577b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:14.443685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:14.444924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 17:29:14.446082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:14.452487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 486/974] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:29:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:14.455157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618723f8-3725-4499-becc-7645210577b9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.462673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f42b7-e333-4a5f-ba1b-5785a34788b3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:14.514865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:14.515546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.515860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.516560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.517666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:14.517991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:14.519622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.540700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 489/975] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:14.543765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.558714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:14.588846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618723f8-3725-4499-becc-7645210577b9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:14.634062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2144b9-1b46-40cd-a11b-565f83125330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:14.634437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2144b9-1b46-40cd-a11b-565f83125330 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.634733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2144b9-1b46-40cd-a11b-565f83125330 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.635015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2144b9-1b46-40cd-a11b-565f83125330 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.635844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2144b9-1b46-40cd-a11b-565f83125330 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:14.636187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2144b9-1b46-40cd-a11b-565f83125330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:14.637672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2144b9-1b46-40cd-a11b-565f83125330 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.682588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 487/976] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Jul 15 17:29:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:14.682788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220cbba-6069-4307-8f5e-85e24fec15e7 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:14.697641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d053-0e66-4226-92da-b2648f242460 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.698249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d053-0e66-4226-92da-b2648f242460 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.698473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91d053-0e66-4226-92da-b2648f242460 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.700366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 488/977] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:14.700896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:14.706501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20b80-cb9e-49a3-b177-f955de221a0e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.707168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20b80-cb9e-49a3-b177-f955de221a0e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.707665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee20b80-cb9e-49a3-b177-f955de221a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.725683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:14.726754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.727190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.727651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.728812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1ee20b80-cb9e-49a3-b177-f955de221a0e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:14.730801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1ee20b80-cb9e-49a3-b177-f955de221a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:14.735807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:14.737577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:14.754628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:14.755785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:14.756960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:14.759269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 490/978] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:14.765945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.833987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 489/979] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:14.855476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20b80-cb9e-49a3-b177-f955de221a0e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.900208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5e269-80f3-450f-87e7-bc12e18ecac2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:14.936336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:14.951203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:14.953068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:14.953068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:14.953068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:14.953780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:14.954096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:14.955554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:14.970868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 491/980] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:14.976979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:14.989497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee20b80-cb9e-49a3-b177-f955de221a0e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:15.013788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:15.026428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0a0e2-a97d-4a9c-bb3b-6f0a892bb14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:15.027099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0a0e2-a97d-4a9c-bb3b-6f0a892bb14c None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:15.027382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0a0e2-a97d-4a9c-bb3b-6f0a892bb14c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:15.027662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0a0e2-a97d-4a9c-bb3b-6f0a892bb14c None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:15.030428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0a0e2-a97d-4a9c-bb3b-6f0a892bb14c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:15.030428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0a0e2-a97d-4a9c-bb3b-6f0a892bb14c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:15.088087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb382-1cad-48c8-aa06-e7db1c404195 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:15.108037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 490/981] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:29:14 2022] POST /identity/v3/services => generated 242 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:15.134448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e762b-28a0-43f4-bcd8-9c9794f6fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:15.135136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e762b-28a0-43f4-bcd8-9c9794f6fafd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:15.135427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e762b-28a0-43f4-bcd8-9c9794f6fafd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:15.135708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e762b-28a0-43f4-bcd8-9c9794f6fafd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:15.136654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e762b-28a0-43f4-bcd8-9c9794f6fafd None admin] RBAC: Adding query filter params (role.id=d3f1418897324c3d88c6f01b75cf98e9, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=812f104fcd164e748c3d6558228d7ef0) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:15.136980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e762b-28a0-43f4-bcd8-9c9794f6fafd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:15.138645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e762b-28a0-43f4-bcd8-9c9794f6fafd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:15.156691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 492/982] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:14 2022] GET /identity/v3/role_assignments?user.id=812f104fcd164e748c3d6558228d7ef0&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=d3f1418897324c3d88c6f01b75cf98e9 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:16.200870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2ed1-dd1c-43d8-a65a-5d71bd7ac07e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:16.201414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2ed1-dd1c-43d8-a65a-5d71bd7ac07e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:16.201414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a2ed1-dd1c-43d8-a65a-5d71bd7ac07e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:16.203547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 491/983] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:16.208522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e8e45-cf4d-47c3-9f52-3fe270bd182f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:16.209494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e8e45-cf4d-47c3-9f52-3fe270bd182f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:16.209978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e8e45-cf4d-47c3-9f52-3fe270bd182f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:16.228780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-db3e8e45-cf4d-47c3-9f52-3fe270bd182f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:16.230302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-db3e8e45-cf4d-47c3-9f52-3fe270bd182f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:16.305604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 493/984] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:16.329598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a2ed1-dd1c-43d8-a65a-5d71bd7ac07e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:16.450446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a2ed1-dd1c-43d8-a65a-5d71bd7ac07e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:16.474631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:16.486773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:16.487570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:16.487854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:16.488128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:16.488802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:16.490176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:16.503918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:16.505152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:16.506355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:16.509744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 492/985] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:16.515083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e8e45-cf4d-47c3-9f52-3fe270bd182f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:16.533608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:16.618846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e8e45-cf4d-47c3-9f52-3fe270bd182f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:16.653647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:16.654324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:16.654618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:16.654893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:16.655633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:16.655940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:16.657332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:16.666956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 494/986] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:16.668127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:16.771505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab96ef-f66d-425d-90dc-ee550117d407 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:16.805513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:16.806218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:16.806504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:16.806783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:16.807489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:16.807795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:16.809255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:16.819023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 493/987] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:16.820274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:16.905466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:16.924911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312baa9f-45d0-490b-a02d-94ed1a7bc535 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:16.961664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca931f-b6b8-4962-9f64-bc97953695df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:16.962326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca931f-b6b8-4962-9f64-bc97953695df None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:16.962616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca931f-b6b8-4962-9f64-bc97953695df None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:16.962894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca931f-b6b8-4962-9f64-bc97953695df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:16.963703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca931f-b6b8-4962-9f64-bc97953695df None admin] RBAC: Adding query filter params (interface=admin, service_id=daa612f2ffa7420095c333a678198ab9, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:16.964017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca931f-b6b8-4962-9f64-bc97953695df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:16.965409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca931f-b6b8-4962-9f64-bc97953695df None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:16.974932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 495/988] 10.222.0.57 () {56 vars in 1249 bytes} [Fri Jul 15 17:29:16 2022] GET /identity/v3/endpoints?service_id=daa612f2ffa7420095c333a678198ab9&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:16.976449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.029542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:17.080233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd640f-25ac-4ee1-b9f8-e0918c164d55 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.113512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c5a4-c32e-43ff-b552-f18f82e820d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.114178 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c5a4-c32e-43ff-b552-f18f82e820d5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.114461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c5a4-c32e-43ff-b552-f18f82e820d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.114734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c5a4-c32e-43ff-b552-f18f82e820d5 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.115413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c5a4-c32e-43ff-b552-f18f82e820d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.116946 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c5a4-c32e-43ff-b552-f18f82e820d5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.124723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 494/989] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:29:16 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:29:17.166765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454aa2f-a540-43da-9f72-d99f3236d0a5 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.167085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454aa2f-a540-43da-9f72-d99f3236d0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.167336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454aa2f-a540-43da-9f72-d99f3236d0a5 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.168825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 496/990] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:17.174677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae112a-525c-41f5-a653-a2a3cdee563a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.174986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae112a-525c-41f5-a653-a2a3cdee563a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.175255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae112a-525c-41f5-a653-a2a3cdee563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.193515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6cae112a-525c-41f5-a653-a2a3cdee563a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:17.194737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6cae112a-525c-41f5-a653-a2a3cdee563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:17.228516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023dcd-ef36-4faf-a225-afa58d2e5e6d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.229072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023dcd-ef36-4faf-a225-afa58d2e5e6d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.229072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2023dcd-ef36-4faf-a225-afa58d2e5e6d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.230764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 497/991] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:17.236902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314830-95fb-48c7-961e-40d1c0b3c97b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.237279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314830-95fb-48c7-961e-40d1c0b3c97b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.237626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314830-95fb-48c7-961e-40d1c0b3c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.255496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2d314830-95fb-48c7-961e-40d1c0b3c97b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:17.256870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2d314830-95fb-48c7-961e-40d1c0b3c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:17.285051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 495/992] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:17.307917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae112a-525c-41f5-a653-a2a3cdee563a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.334838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 498/993] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:17.356335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314830-95fb-48c7-961e-40d1c0b3c97b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.361158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:17.421758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae112a-525c-41f5-a653-a2a3cdee563a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.433854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:17.458458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.459250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.459534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.459785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314830-95fb-48c7-961e-40d1c0b3c97b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.459892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.460438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.461854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:17.473830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:17.474850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:17.475838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 496/994] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:17.480777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.501641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.502442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.502889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.503172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.503850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.505244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:17.518300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:17.519414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:17.520248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:17.524467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 499/995] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:17.525473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.588545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b208f-d799-4961-91b1-768d290b05b4 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.599524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:17.625254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.626039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.626436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.626721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.627435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] RBAC: Adding query filter params (name=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:17.627742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.629468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.629832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08efc95-9936-4a35-af28-e26a64f0e7d1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.639636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 497/996] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:17.640866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.668407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.669099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.669385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.669662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.670365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] RBAC: Adding query filter params (name=image) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:17.670740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.672102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.677231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 500/997] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:17.682624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.685450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:17.758435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea5a3-057e-4de2-a5ab-f95cafb0f3ac None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.788665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac600bd-70c6-43d0-9fec-5ee88af60cbb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.796519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.797219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.797509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.797790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.798864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:17.799179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.800907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.806911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 498/998] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:17.812294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:17.826397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.827158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.827447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.827759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.828470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] RBAC: Adding query filter params (type=image) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:17.828821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.830191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.835146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 501/999] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:17.862166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:17.934069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a2339-46b9-4662-95c6-376f220c7938 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:17.972203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242fcc6-981a-4e24-8e0d-5b15ab65ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:17.973217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242fcc6-981a-4e24-8e0d-5b15ab65ba34 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:17.973649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242fcc6-981a-4e24-8e0d-5b15ab65ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:17.974072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242fcc6-981a-4e24-8e0d-5b15ab65ba34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:17.975039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242fcc6-981a-4e24-8e0d-5b15ab65ba34 None admin] RBAC: Adding query filter params (interface=public, service_id=47f4301e8391431f813a91d0e641ce92, region_id=RegionOne) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:17.975496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242fcc6-981a-4e24-8e0d-5b15ab65ba34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:17.977070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242fcc6-981a-4e24-8e0d-5b15ab65ba34 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:17.988368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 499/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/endpoints?service_id=47f4301e8391431f813a91d0e641ce92&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:17.989706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:18.057766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:18.093394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2791f75-4e77-4fe4-bce2-5619703fe623 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:18.127439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cb3e5-f579-410f-8583-bc6f715750ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:18.128021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb3e5-f579-410f-8583-bc6f715750ba None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:18.128301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb3e5-f579-410f-8583-bc6f715750ba None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:18.128570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb3e5-f579-410f-8583-bc6f715750ba None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:18.129274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cb3e5-f579-410f-8583-bc6f715750ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:18.130623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cb3e5-f579-410f-8583-bc6f715750ba None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:18.137298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 502/1001] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:29:17 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:29:19.161287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe9241-12a3-40f0-af4b-946a7318fc13 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:19.161804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe9241-12a3-40f0-af4b-946a7318fc13 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:19.161882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe9241-12a3-40f0-af4b-946a7318fc13 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:19.163373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 500/1002] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:19.168383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ef986-f84a-4df3-9c33-955df2f36063 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:19.169058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ef986-f84a-4df3-9c33-955df2f36063 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:19.169058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ef986-f84a-4df3-9c33-955df2f36063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:19.185983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-488ef986-f84a-4df3-9c33-955df2f36063 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:19.187664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-488ef986-f84a-4df3-9c33-955df2f36063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:19.260221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 503/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:19.283009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe9241-12a3-40f0-af4b-946a7318fc13 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:19.382251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:19.386409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe9241-12a3-40f0-af4b-946a7318fc13 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:19.421408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:19.422140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:19.422430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:19.422710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:19.423366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:19.424772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:19.436749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:19.437911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:19.438945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 501/1004] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Jul 15 17:29:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:19.445768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ef986-f84a-4df3-9c33-955df2f36063 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:19.506285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:19.549312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ef986-f84a-4df3-9c33-955df2f36063 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:19.584753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:19.585430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:19.585717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:19.585994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:19.586957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:19.587068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:19.588362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:19.598304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 504/1005] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:19.599847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:19.702113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d312b-f354-4a78-974e-75fa2473e313 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:19.735721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3710da-23c9-4004-8b7f-7b956c02fa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:19.736390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3710da-23c9-4004-8b7f-7b956c02fa17 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:19.736706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3710da-23c9-4004-8b7f-7b956c02fa17 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:19.736990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3710da-23c9-4004-8b7f-7b956c02fa17 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:19.737696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3710da-23c9-4004-8b7f-7b956c02fa17 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:19.738006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3710da-23c9-4004-8b7f-7b956c02fa17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:19.739388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3710da-23c9-4004-8b7f-7b956c02fa17 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:19.744485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 502/1006] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Jul 15 17:29:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:19.750045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:19.853093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400185b1-975a-46f0-8d9b-2d18a0f42a3d None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:19.878937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:19.890306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1365f0-4e3d-4953-a2ed-63eebfb3bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:19.890977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1365f0-4e3d-4953-a2ed-63eebfb3bd52 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:19.891264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1365f0-4e3d-4953-a2ed-63eebfb3bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:19.891542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1365f0-4e3d-4953-a2ed-63eebfb3bd52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:19.892292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1365f0-4e3d-4953-a2ed-63eebfb3bd52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:19.893682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1365f0-4e3d-4953-a2ed-63eebfb3bd52 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:19.979489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 505/1007] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:29:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:20.187537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9f9dc-44a8-4bcd-b989-73c5620a2d58 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:20.188060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9f9dc-44a8-4bcd-b989-73c5620a2d58 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:20.188138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9f9dc-44a8-4bcd-b989-73c5620a2d58 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:20.189687 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 503/1008] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:20.194591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bbf27-7412-456c-bc5e-60e7fe3db674 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:20.194991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bbf27-7412-456c-bc5e-60e7fe3db674 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:20.195173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bbf27-7412-456c-bc5e-60e7fe3db674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:20.212403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3b8bbf27-7412-456c-bc5e-60e7fe3db674 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:20.213668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3b8bbf27-7412-456c-bc5e-60e7fe3db674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:20.296645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 506/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:20.318055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9f9dc-44a8-4bcd-b989-73c5620a2d58 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:20.349293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:20.425276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9f9dc-44a8-4bcd-b989-73c5620a2d58 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:20.463116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:20.463790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:20.464079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:20.464360 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:20.465131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:20.466520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:20.472010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56868b-7ab0-4113-8a5e-cbb196fc01fe None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:20.472302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56868b-7ab0-4113-8a5e-cbb196fc01fe None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:20.472582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56868b-7ab0-4113-8a5e-cbb196fc01fe None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:20.474007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 507/1010] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:20.479208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9f378-4a5b-4030-b9d4-2760850dd0e9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:20.481384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9f378-4a5b-4030-b9d4-2760850dd0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:20.481384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9f378-4a5b-4030-b9d4-2760850dd0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:20.497042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-14c9f378-4a5b-4030-b9d4-2760850dd0e9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:20.498161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-14c9f378-4a5b-4030-b9d4-2760850dd0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:20.546530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 504/1011] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:29:20 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:29:20.577215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 508/1012] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:20 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:20.600891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:20.702894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:20.715583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77775a4-7b11-4eb7-80a6-77daac1ce065 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:20.751950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:20.752704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:20.752993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:20.753271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:20.753920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:20.755502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:20.769285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:20.770343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:20.771191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 505/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:20.779102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9f378-4a5b-4030-b9d4-2760850dd0e9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:20.815437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:20.883106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9f378-4a5b-4030-b9d4-2760850dd0e9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:20.917663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:20.918324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:20.918606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:20.918877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:20.919574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] RBAC: Adding query filter params (name=compute) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:20.919874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:20.921911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:20.931429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 509/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:29:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:20.932558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:21.032586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355176d-ef5f-4e19-845d-b4c1ca75db5f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:21.065775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6b703-3904-4a98-a81b-ef574527e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:21.066443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6b703-3904-4a98-a81b-ef574527e2bd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:21.066727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6b703-3904-4a98-a81b-ef574527e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:21.067004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6b703-3904-4a98-a81b-ef574527e2bd None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:21.067707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6b703-3904-4a98-a81b-ef574527e2bd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:21.068028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6b703-3904-4a98-a81b-ef574527e2bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:21.069410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6b703-3904-4a98-a81b-ef574527e2bd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:21.079878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 506/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Jul 15 17:29:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:21.081141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:21.181365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:21.185011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a06949-06a4-456d-8020-bdb4f08cd3de None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:21.218950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d832c-319e-4420-8dfe-9d285e1d54a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:21.219714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d832c-319e-4420-8dfe-9d285e1d54a2 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:21.220088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d832c-319e-4420-8dfe-9d285e1d54a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:21.220668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d832c-319e-4420-8dfe-9d285e1d54a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:21.221649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d832c-319e-4420-8dfe-9d285e1d54a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:21.223476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d832c-319e-4420-8dfe-9d285e1d54a2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:21.324634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 510/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:29:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:29:22.040137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb2d54-f9e8-4347-9bbb-51d29df4ad5d None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:22.040731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb2d54-f9e8-4347-9bbb-51d29df4ad5d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:22.040731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb2d54-f9e8-4347-9bbb-51d29df4ad5d None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:22.042261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 507/1017] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:22.048055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690d0bb-2e68-4b07-8cfb-54c6327fd515 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:22.048420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690d0bb-2e68-4b07-8cfb-54c6327fd515 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:22.048802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690d0bb-2e68-4b07-8cfb-54c6327fd515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:22.066157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5690d0bb-2e68-4b07-8cfb-54c6327fd515 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:22.067348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5690d0bb-2e68-4b07-8cfb-54c6327fd515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:22.142256 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 511/1018] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:22.165396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb2d54-f9e8-4347-9bbb-51d29df4ad5d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:22.184422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:22.282146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb2d54-f9e8-4347-9bbb-51d29df4ad5d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:22.317610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea6b6e-0a2d-46b9-84c2-ab8f94fae611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:22.318407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6b6e-0a2d-46b9-84c2-ab8f94fae611 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:22.318696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6b6e-0a2d-46b9-84c2-ab8f94fae611 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:22.318975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea6b6e-0a2d-46b9-84c2-ab8f94fae611 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:22.320148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea6b6e-0a2d-46b9-84c2-ab8f94fae611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:22.321637 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea6b6e-0a2d-46b9-84c2-ab8f94fae611 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:22.328836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 508/1019] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:22.331416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690d0bb-2e68-4b07-8cfb-54c6327fd515 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:22.414453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:22.433265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690d0bb-2e68-4b07-8cfb-54c6327fd515 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:22.469766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5924b76-8071-413c-96ea-58e99ec50d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:22.470445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924b76-8071-413c-96ea-58e99ec50d93 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:22.470716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924b76-8071-413c-96ea-58e99ec50d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:22.470999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924b76-8071-413c-96ea-58e99ec50d93 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:22.471714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5924b76-8071-413c-96ea-58e99ec50d93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:22.473188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5924b76-8071-413c-96ea-58e99ec50d93 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:22.567018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 512/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:29:22 2022] POST /identity/v3/projects => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:29:22.588405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8c12-b7c1-4296-bc4a-8e92cfceac72 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:22.588405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8c12-b7c1-4296-bc4a-8e92cfceac72 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:22.588405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8c12-b7c1-4296-bc4a-8e92cfceac72 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:22.588405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 509/1021] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:22.592915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b447471-e53c-4251-bc81-3c407774d0e6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:22.593222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b447471-e53c-4251-bc81-3c407774d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:22.593502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b447471-e53c-4251-bc81-3c407774d0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:22.611342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5b447471-e53c-4251-bc81-3c407774d0e6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:22.612556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5b447471-e53c-4251-bc81-3c407774d0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:22.687064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 513/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:22.710188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8c12-b7c1-4296-bc4a-8e92cfceac72 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:22.802684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:22.819418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8c12-b7c1-4296-bc4a-8e92cfceac72 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:22.854607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:22.855324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:22.855610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:22.855889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:22.856548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:22.858007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:22.869733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:22.870750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:22.871601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 510/1023] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Jul 15 17:29:22 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:22.879384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b447471-e53c-4251-bc81-3c407774d0e6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:22.945462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:22.983826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b447471-e53c-4251-bc81-3c407774d0e6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:23.019050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:23.020489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.020802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.021085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.021798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] RBAC: Adding query filter params (name=image) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:23.022104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:23.023475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:23.034139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 514/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:23.035355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:23.138079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46f0de-376c-4694-92cf-c2b09c7469f8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:23.171784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:23.172447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.172763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.173045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.173758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] RBAC: Adding query filter params (type=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:23.174067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:23.175444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:23.185652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 511/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:23.187455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:23.299830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09628b5-712b-422f-92f4-7eb774a4cf16 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:23.327705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:23.336983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60ea96-e77b-44bd-8d23-ff72caaab2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:23.337681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60ea96-e77b-44bd-8d23-ff72caaab2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.337960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60ea96-e77b-44bd-8d23-ff72caaab2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.338217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60ea96-e77b-44bd-8d23-ff72caaab2a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.339012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60ea96-e77b-44bd-8d23-ff72caaab2a1 None admin] RBAC: Adding query filter params (interface=public, service_id=cb58a5717d084e0d880d252267e61775, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:23.339328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60ea96-e77b-44bd-8d23-ff72caaab2a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:23.340728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60ea96-e77b-44bd-8d23-ff72caaab2a1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:23.350663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 515/1026] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:29:23 2022] GET /identity/v3/endpoints?service_id=cb58a5717d084e0d880d252267e61775&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:23.352251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:23.369516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:23.426538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe655c-488e-4270-9d36-f1ea0fa5c5cc None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.426743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe655c-488e-4270-9d36-f1ea0fa5c5cc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.427025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe655c-488e-4270-9d36-f1ea0fa5c5cc None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.428404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 516/1027] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:23.433746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb030d-1c78-47ed-b0df-bc03498d5a4f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.434045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb030d-1c78-47ed-b0df-bc03498d5a4f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.434330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb030d-1c78-47ed-b0df-bc03498d5a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.451343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1fcb030d-1c78-47ed-b0df-bc03498d5a4f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:23.452622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1fcb030d-1c78-47ed-b0df-bc03498d5a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:23.459583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450729b-b089-4cef-a00b-313c7a25a350 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:23.499532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:23.500199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.500489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.500808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.501536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:23.502876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:23.509659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 512/1028] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:29:23 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:29:23.522666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 517/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:23.547040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:23.640854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:23.665883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a1ac7-e6ab-423d-8694-2d0ae9ac1a5d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:23.702607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:23.703340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.703627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.703903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.714319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:23.722754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:23.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:23.725075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 513/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 15 17:29:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:23.730676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb030d-1c78-47ed-b0df-bc03498d5a4f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:23.832848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb030d-1c78-47ed-b0df-bc03498d5a4f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:23.856027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:23.868001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:23.868707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:23.868998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:23.869287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:23.870046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:23.870359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:23.871743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:23.885172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 518/1031] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 15 17:29:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:23.887243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:23.993620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3ff26-c7c2-410e-b3a6-bc65ee10e72d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:24.028117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:24.028953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.029244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.029526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.032311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:24.040908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:24.042381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:24.060069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:24.061671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:24.068192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:24.068192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:24.068192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:24.068192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:24.068192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 514/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:24.070242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:24.173642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53d186-a7a9-4913-9c45-aa89b0d41cb3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:24.207728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:24.208403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.208803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.209092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.209814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:24.210120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:24.211487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:24.228364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 519/1033] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:24.229806 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:24.340019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf438624-9efb-48dc-b45e-9fb1cc44a66b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:24.374019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:24.374808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.375095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.375372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.382926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:24.384380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:24.398407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:24.399372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:24.406169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 515/1034] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:29:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:24.408443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:24.455984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:24.515434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b47db-2343-4e35-a743-cfc44e7c0a50 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:24.549063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6182b-892e-4a8b-985f-0eaddc3b510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:24.549732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6182b-892e-4a8b-985f-0eaddc3b510f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.550018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6182b-892e-4a8b-985f-0eaddc3b510f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.550296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6182b-892e-4a8b-985f-0eaddc3b510f None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.551136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6182b-892e-4a8b-985f-0eaddc3b510f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:24.551446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6182b-892e-4a8b-985f-0eaddc3b510f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:24.552886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6182b-892e-4a8b-985f-0eaddc3b510f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:24.572541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 520/1035] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:29:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:24.578725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:24.631295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:24.640654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6258e-a26a-4f70-9296-abbd69ac5bcf None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.640654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6258e-a26a-4f70-9296-abbd69ac5bcf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.640654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6258e-a26a-4f70-9296-abbd69ac5bcf None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.640654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 521/1036] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:24.645324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c13e3e-df8b-4857-9945-f88d4115df82 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.645607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c13e3e-df8b-4857-9945-f88d4115df82 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.645881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c13e3e-df8b-4857-9945-f88d4115df82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.663512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-97c13e3e-df8b-4857-9945-f88d4115df82 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:24.664740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-97c13e3e-df8b-4857-9945-f88d4115df82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:24.692209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11960d-2507-4e74-a009-04f6c07ae60c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:24.727656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:24.728313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.728615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.728987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.740646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 522/1037] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:24.761276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c13e3e-df8b-4857-9945-f88d4115df82 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:24.794104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:24.808254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:24.810455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:24.865134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:24.866053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c13e3e-df8b-4857-9945-f88d4115df82 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:24.901588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:24.902638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:24.903594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:24.903884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:24.905135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:24.906566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:24.912996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 523/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:24.914579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:24.925393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 516/1039] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:24.926519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:25.023718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:25.027141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5c638-1895-4a02-86eb-c1812102e770 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:25.036955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f11dc1-6f24-492e-961a-2105c95bf20b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:25.062939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:25.063800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.064086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.064710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.065434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:25.066866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:25.072209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:25.073132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.073511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.073890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.084964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:25.086374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:25.094944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:25.104213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:25.105325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:25.113366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:25.113366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:25.113366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:25.113366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:25.113366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 517/1040] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:25.113366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:25.171566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 524/1041] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:29:24 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:29:25.227409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d07ed-ef9d-4f95-abc4-bc77b302226c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:25.273331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:25.274023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.274280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.274559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.275558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:25.275864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:25.277305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:25.291530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 518/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:25.297233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:25.412698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04983eba-88c5-4be4-ba49-374e24fc9bba None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:25.447880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe914d-bcfb-40bb-bf92-4dbf21c3587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:25.448707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe914d-bcfb-40bb-bf92-4dbf21c3587f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.448991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe914d-bcfb-40bb-bf92-4dbf21c3587f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.449267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe914d-bcfb-40bb-bf92-4dbf21c3587f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.456679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe914d-bcfb-40bb-bf92-4dbf21c3587f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:25.458329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe914d-bcfb-40bb-bf92-4dbf21c3587f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-5dfe914d-bcfb-40bb-bf92-4dbf21c3587f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:25.473141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:25.474805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:25.476578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 525/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:25.486525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:25.555351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c2e9-5fa9-4a02-be44-673d65757c8f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.555700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c2e9-5fa9-4a02-be44-673d65757c8f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.555976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c2e9-5fa9-4a02-be44-673d65757c8f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.557512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 526/1044] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:25.559099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:25.563648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce8ae4-f3c9-431c-a3a3-5c2f163d72e1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.563942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce8ae4-f3c9-431c-a3a3-5c2f163d72e1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.564228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce8ae4-f3c9-431c-a3a3-5c2f163d72e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.582308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-29ce8ae4-f3c9-431c-a3a3-5c2f163d72e1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:25.583522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-29ce8ae4-f3c9-431c-a3a3-5c2f163d72e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:25.599215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:25.606313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e4da0-9c21-4a64-aa79-f5e59b5c59d8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:25.644520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:25.645287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.645514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.645796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.646607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:25.646920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:25.648428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:25.661823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 519/1045] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:29:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:25.663203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 527/1046] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:25.673740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:25.687941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce8ae4-f3c9-431c-a3a3-5c2f163d72e1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:25.787977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7d017-5ee0-4000-b8e7-ae383e4a882b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:25.790438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce8ae4-f3c9-431c-a3a3-5c2f163d72e1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:25.808332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:25.823652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:25.824314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.824628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.825612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.825871 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] RBAC: Adding query filter params (role.id=d3f1418897324c3d88c6f01b75cf98e9, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=3ba11eb5837a4567bd77f3fcb0017ea7) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:25.826119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:25.827603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:25.828120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:25.828389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:25.828700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:25.828977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] PATH_INFO: `/v3/services/image` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:25.829625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:25.831014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:25.833750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 520/1047] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:25 2022] GET /identity/v3/role_assignments?user.id=3ba11eb5837a4567bd77f3fcb0017ea7&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=d3f1418897324c3d88c6f01b75cf98e9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:25.844035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:25.845152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 17:29:25.845975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:25.848972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 528/1048] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Jul 15 17:29:25 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:25.851309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:25.970019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b6f1-974a-4ef8-88cd-4ce30ae748f8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:26.006765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:26.007446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:26.007736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:26.008018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:26.008797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:26.009081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:26.010472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:26.015658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 521/1049] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:26.021503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:26.035381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:26.127261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eff66-2384-4735-8ba9-c8afd481b9af None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:26.161817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318439c0-351a-4d41-9f29-a03044adcd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:26.162473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318439c0-351a-4d41-9f29-a03044adcd61 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:26.162755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318439c0-351a-4d41-9f29-a03044adcd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:26.163029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318439c0-351a-4d41-9f29-a03044adcd61 None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:26.163729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318439c0-351a-4d41-9f29-a03044adcd61 None admin] RBAC: Adding query filter params (type=image) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:26.164033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318439c0-351a-4d41-9f29-a03044adcd61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:26.165505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318439c0-351a-4d41-9f29-a03044adcd61 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:26.175348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 529/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:26.176569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:26.231152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:26.306202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3353ee8-707e-4de3-8ba3-325d707a32f2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:26.342491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c822b-08cb-4e9a-abbd-0c057a0d5d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:26.343158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c822b-08cb-4e9a-abbd-0c057a0d5d53 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:26.343448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c822b-08cb-4e9a-abbd-0c057a0d5d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:26.343732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c822b-08cb-4e9a-abbd-0c057a0d5d53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:26.344480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c822b-08cb-4e9a-abbd-0c057a0d5d53 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:26.345937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c822b-08cb-4e9a-abbd-0c057a0d5d53 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:26.428048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 522/1051] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:29:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:27.286863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2972f-5aaa-47e8-96c3-9d07333f070f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:27.287404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2972f-5aaa-47e8-96c3-9d07333f070f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:27.287404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2972f-5aaa-47e8-96c3-9d07333f070f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:27.288953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 530/1052] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:27.293901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe98cc-1b1a-408e-b1c3-ac84883d3eb7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:27.294199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe98cc-1b1a-408e-b1c3-ac84883d3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:27.294481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe98cc-1b1a-408e-b1c3-ac84883d3eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:27.312264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bcfe98cc-1b1a-408e-b1c3-ac84883d3eb7 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:27.313469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bcfe98cc-1b1a-408e-b1c3-ac84883d3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:27.387385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 523/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:27.410925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2972f-5aaa-47e8-96c3-9d07333f070f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:27.489686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:27.516921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2972f-5aaa-47e8-96c3-9d07333f070f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:27.552615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:27.553425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:27.553707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:27.553979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:27.565185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:27.573162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:27.574184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:27.575011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 531/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:29:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:27.582715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe98cc-1b1a-408e-b1c3-ac84883d3eb7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:27.612077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:27.684261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe98cc-1b1a-408e-b1c3-ac84883d3eb7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:27.718983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8576d-c6b3-49c2-98a0-b0312c72ba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:27.719652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8576d-c6b3-49c2-98a0-b0312c72ba21 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:27.719934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8576d-c6b3-49c2-98a0-b0312c72ba21 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:27.720208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8576d-c6b3-49c2-98a0-b0312c72ba21 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:27.720984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8576d-c6b3-49c2-98a0-b0312c72ba21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:27.721292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8576d-c6b3-49c2-98a0-b0312c72ba21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:27.722680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8576d-c6b3-49c2-98a0-b0312c72ba21 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:27.735895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 524/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:29:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:27.737129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:27.836555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc1ec9-f83c-4df2-b25a-a563003b8cdb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:27.870127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2c3b-0df1-465d-8112-81f339522312 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:27.870447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2c3b-0df1-465d-8112-81f339522312 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:27.870577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:27.870675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2c3b-0df1-465d-8112-81f339522312 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:27.871412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:27.871693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:27.871968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] PATH_INFO: `/v3/users/b551638d1ff242de91e2e4c6f11a8b91` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:27.872309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 525/1056] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:27.874977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:27.878141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc4777-e7d9-4cca-903c-2bb9fccf25d4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:27.878440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc4777-e7d9-4cca-903c-2bb9fccf25d4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:27.878727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc4777-e7d9-4cca-903c-2bb9fccf25d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:27.888298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] RBAC: Authorizing `identity:get_user(user_id=b551638d1ff242de91e2e4c6f11a8b91)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:27.889841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:27.891573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 532/1057] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:27 2022] GET /identity/v3/users/b551638d1ff242de91e2e4c6f11a8b91 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:27.900151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-aadc4777-e7d9-4cca-903c-2bb9fccf25d4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:27.900151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aadc4777-e7d9-4cca-903c-2bb9fccf25d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:27.902756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:27.974687 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 526/1058] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:27.997092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc4777-e7d9-4cca-903c-2bb9fccf25d4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.009636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489122-19a3-402f-8db6-fb579cabe2fd None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.023547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.050190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.051018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.051600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.051893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.105796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc4777-e7d9-4cca-903c-2bb9fccf25d4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.118267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.128323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.129910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:28.144479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.145409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.145728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.145977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.157701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:28.165695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:28.166662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:28.169483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 527/1059] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 15 17:29:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:28.173417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.243551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 533/1060] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 17:29:28.247735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.266682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.284867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01522e8-7b9a-4368-90e5-1db9960f1677 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.321405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.322853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.323148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.323433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.324194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:28.324507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.325944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:28.334393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 528/1061] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:28.339811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.348614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.358297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ce759-837d-4ea6-bbe6-6e7146d6cf18 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.395495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.396365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.396677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.397353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.398571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] RBAC: Authorizing `identity:get_project(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.400254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:28.402163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 534/1062] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:28.408529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.455178 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cf1ea-5962-45b2-a6c3-e952602c78c2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.489863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6af7e2-c123-40a4-b4cc-68de1b55c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.497165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6af7e2-c123-40a4-b4cc-68de1b55c00a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.497455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6af7e2-c123-40a4-b4cc-68de1b55c00a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.497740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6af7e2-c123-40a4-b4cc-68de1b55c00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.513535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6af7e2-c123-40a4-b4cc-68de1b55c00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.515039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6af7e2-c123-40a4-b4cc-68de1b55c00a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:28.517955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaead0-4d0e-48a8-8768-bae1d8512746 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.518201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3f6af7e2-c123-40a4-b4cc-68de1b55c00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:28.533439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:28.534533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:28.535767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:28.535767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:28.535767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:28.535767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:28.535767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 529/1063] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:28.537554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8e0fa-e91e-4406-a0ca-4d0388f26101 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.537847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8e0fa-e91e-4406-a0ca-4d0388f26101 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.538130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8e0fa-e91e-4406-a0ca-4d0388f26101 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.539508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 530/1064] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:28.540256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8e0fa-e91e-4406-a0ca-4d0388f26101 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.558640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8809b-74f2-4543-a4ff-006a02b8e015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.559294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8809b-74f2-4543-a4ff-006a02b8e015 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.559581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8809b-74f2-4543-a4ff-006a02b8e015 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.559857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8809b-74f2-4543-a4ff-006a02b8e015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.560768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8809b-74f2-4543-a4ff-006a02b8e015 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc, user.id=b551638d1ff242de91e2e4c6f11a8b91) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:28.561620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8809b-74f2-4543-a4ff-006a02b8e015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.563040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8809b-74f2-4543-a4ff-006a02b8e015 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:28.568475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 535/1065] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/role_assignments?user.id=b551638d1ff242de91e2e4c6f11a8b91&scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:28.569664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c39b-8d6a-4938-a614-4214d03d33f6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.569952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c39b-8d6a-4938-a614-4214d03d33f6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.570228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c39b-8d6a-4938-a614-4214d03d33f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.596324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4927c39b-8d6a-4938-a614-4214d03d33f6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:28.597606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4927c39b-8d6a-4938-a614-4214d03d33f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:28.620550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.657336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8e0fa-e91e-4406-a0ca-4d0388f26101 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.679947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 536/1066] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:28.695806 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.696478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.696839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.697119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.697853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:28.698161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.699571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:28.703285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927c39b-8d6a-4938-a614-4214d03d33f6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.716622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 531/1067] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:28.718208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.777857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.815873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927c39b-8d6a-4938-a614-4214d03d33f6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.827070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c751f36-85dc-44a8-8dbb-c0e0d817ac6f None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:28.852317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.852989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.853274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.853550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.863325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:28.864119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:28.864410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:28.864712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:28.872641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.874572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:28.879879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:28.889714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:28.894382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:28.895319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 532/1068] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:28.897443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:28.933141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:28.934999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.003131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55d1d1-c6f4-4eac-8fef-8c2677dcc127 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.019335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:29.041552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.042226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.042618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.042932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.043805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:29.044117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.045661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.048710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 537/1069] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 17:29:29.050796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:29.066692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 533/1070] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:29:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:29.072221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:29.114437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:29.160983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f3852-fae9-4f96-9bff-5ecf590e7cbc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.187260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d88cf-44b0-4f4e-8a01-d85f2d9bec1c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.196937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.197827 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.198182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.198527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.209132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.210675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:29.228483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:29.229623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:29.230731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:29.230731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:29.230731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:29.230731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:29.236687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 538/1071] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:29.237893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:29.244433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.245291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.245579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.245858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.256820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.258337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:29.276609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:29.277524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:29.278419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:29.278419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:29.278419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:29.278419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:29.282993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 534/1072] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:29.283857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:29.313246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:29.346382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59459e06-aa8d-4fd9-b302-496b30d7e3b2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.355002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:29.385984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.386954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.387488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.388129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.389402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfdd7a-9b52-4d34-a22c-c70ee0aada63 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.389833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:29.389833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.391091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.403235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 539/1073] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:29.434203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.434807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.435097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.435377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.436778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:29.437093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.438527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.451535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 535/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:29.457316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:29.570015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64898c0-c92f-4f1a-8013-0bae2ae2acb1 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.607296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.608115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.608400 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.608750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.615655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.617271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:29.632484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:29.633463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:29.634360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 540/1075] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:29.639931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:29.734589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:29.748842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70d18d-a6a5-4c98-a9a1-6b6989871e92 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.782203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28455c1-d968-48c4-83a5-21bb5e8cf36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.782851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28455c1-d968-48c4-83a5-21bb5e8cf36a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.783135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28455c1-d968-48c4-83a5-21bb5e8cf36a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.783409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28455c1-d968-48c4-83a5-21bb5e8cf36a None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.784186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28455c1-d968-48c4-83a5-21bb5e8cf36a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:29.784491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28455c1-d968-48c4-83a5-21bb5e8cf36a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.786005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28455c1-d968-48c4-83a5-21bb5e8cf36a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:29.797671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 536/1076] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:29.803803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:29.830199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:29.900262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec060-78c7-4ec5-81e8-55dc868efcaf None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:29.948091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795009ae-5154-48e5-a053-3510670b54be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:29.949009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795009ae-5154-48e5-a053-3510670b54be None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:29.949292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795009ae-5154-48e5-a053-3510670b54be None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:29.949565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795009ae-5154-48e5-a053-3510670b54be None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/3ba11eb5837a4567bd77f3fcb0017ea7/roles/d3f1418897324c3d88c6f01b75cf98e9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:29.952550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795009ae-5154-48e5-a053-3510670b54be None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=3ba11eb5837a4567bd77f3fcb0017ea7, role_id=d3f1418897324c3d88c6f01b75cf98e9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:29.954044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795009ae-5154-48e5-a053-3510670b54be None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:30.027773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 541/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:29:29 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/3ba11eb5837a4567bd77f3fcb0017ea7/roles/d3f1418897324c3d88c6f01b75cf98e9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:29:30.582078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ff04-d32c-4f40-bad3-fd5e0eb6232f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:30.582634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ff04-d32c-4f40-bad3-fd5e0eb6232f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:30.582634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7ff04-d32c-4f40-bad3-fd5e0eb6232f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:30.584083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 537/1078] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:30.589644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae74fd-eec7-4cf4-b8ae-17b59dab4d42 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:30.590105 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae74fd-eec7-4cf4-b8ae-17b59dab4d42 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:30.590198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae74fd-eec7-4cf4-b8ae-17b59dab4d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:30.607121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-05ae74fd-eec7-4cf4-b8ae-17b59dab4d42 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:30.608958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-05ae74fd-eec7-4cf4-b8ae-17b59dab4d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:30.677092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 542/1079] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:30.698758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ff04-d32c-4f40-bad3-fd5e0eb6232f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:30.799296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7ff04-d32c-4f40-bad3-fd5e0eb6232f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:30.831193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:30.832024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:30.832363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:30.832581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:30.842657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:30.850037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:30.851155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:30.852014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 538/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:30.857374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae74fd-eec7-4cf4-b8ae-17b59dab4d42 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:30.875700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:30.965545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae74fd-eec7-4cf4-b8ae-17b59dab4d42 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:30.999309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:30.999971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:31.000259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:31.000533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:31.001309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:31.001618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:31.002979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:31.011098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 543/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:31.016854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:31.025758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:31.116045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e910e-2c6a-49b8-947e-82a68f5e355b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:31.152982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:31.153863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:31.154150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:31.154935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] PATH_INFO: `/v3/users/b551638d1ff242de91e2e4c6f11a8b91` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:31.156535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] RBAC: Authorizing `identity:get_user(user_id=b551638d1ff242de91e2e4c6f11a8b91)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:31.158640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:31.160470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 539/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:31 2022] GET /identity/v3/users/b551638d1ff242de91e2e4c6f11a8b91 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:31.166276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:31.270948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:31.292626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c81eb-52c9-4c6f-b8c5-9c818d2213be None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:31.325736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb8fac-d42c-4062-9fa3-7c2de368b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:31.326587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb8fac-d42c-4062-9fa3-7c2de368b7be None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:31.326867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb8fac-d42c-4062-9fa3-7c2de368b7be None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:31.327137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb8fac-d42c-4062-9fa3-7c2de368b7be None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:31.328307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb8fac-d42c-4062-9fa3-7c2de368b7be None admin] RBAC: Authorizing `identity:get_project(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:31.329875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb8fac-d42c-4062-9fa3-7c2de368b7be None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:31.331564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 544/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:31 2022] GET /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:31.337674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:31.410544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:31.434332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9073eb-2c4e-4fb7-9a17-cb1939316e52 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:31.467456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ef59c-dce5-46e6-aae3-99d24335c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:31.468332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ef59c-dce5-46e6-aae3-99d24335c66c None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:31.468634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ef59c-dce5-46e6-aae3-99d24335c66c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:31.468910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ef59c-dce5-46e6-aae3-99d24335c66c None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc/users/b551638d1ff242de91e2e4c6f11a8b91/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:31.470969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ef59c-dce5-46e6-aae3-99d24335c66c None admin] RBAC: Authorizing `identity:create_grant(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc, user_id=b551638d1ff242de91e2e4c6f11a8b91, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:31.472391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ef59c-dce5-46e6-aae3-99d24335c66c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:31.533213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 540/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:29:31 2022] PUT /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc/users/b551638d1ff242de91e2e4c6f11a8b91/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:29:32.064865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8b880-df8d-466c-9517-6571b2911a72 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:32.065411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8b880-df8d-466c-9517-6571b2911a72 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:32.065411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8b880-df8d-466c-9517-6571b2911a72 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:32.066811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 545/1085] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:32.072114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e446f-788a-400e-9791-fddfbcbcbce7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:32.072501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e446f-788a-400e-9791-fddfbcbcbce7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:32.072715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e446f-788a-400e-9791-fddfbcbcbce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:32.089486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-db7e446f-788a-400e-9791-fddfbcbcbce7 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:32.090661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db7e446f-788a-400e-9791-fddfbcbcbce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:32.156511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 541/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:32.179489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8b880-df8d-466c-9517-6571b2911a72 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:32.285672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8b880-df8d-466c-9517-6571b2911a72 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:32.319439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:32.320223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:32.320508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:32.320810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:32.331341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:32.339368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:32.341881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:32.342734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 546/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Jul 15 17:29:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:32.346663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e446f-788a-400e-9791-fddfbcbcbce7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:32.430066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:32.466283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e446f-788a-400e-9791-fddfbcbcbce7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:32.500258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:32.500963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:32.501253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:32.501529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:32.502270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:32.502577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:32.503960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:32.511730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 542/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 15 17:29:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:32.517257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:32.526145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:32.625711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b032f38-0745-466d-a4b2-6bcbfb83ff66 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:32.661745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:32.662536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:32.662822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:32.663099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:32.673616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:32.675056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:32.692121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:32.693203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:32.694561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:32.694561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:32.694561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:32.694561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:32.694561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 547/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:32.699118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:32.799428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cf601-6b69-467e-9fee-c5b4e4a35179 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:32.823413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:32.831955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:32.832635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:32.835919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:32.835919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:32.835919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:32.835919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:32.835919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:32.848407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 543/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:32.854028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:32.913867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:32.960971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcb929-ab0c-4c80-87e3-0307b0957687 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:32.994373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:32.995157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:32.995448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:32.995725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.004185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:33.005651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:29:33.019909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:33.021090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:33.022384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 548/1091] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Jul 15 17:29:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:33.028983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:33.146529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8474a-b5df-4266-8186-b9625d4789a3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:33.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:33.185001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:33.185285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:33.185559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.186420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:33.186728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:33.188158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:33.208636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 544/1092] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Jul 15 17:29:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:33.214222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:33.319519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dd888-e621-4fd8-8a2f-2eba0d5c946a None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:33.337284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:33.357368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ad8e-335b-4db5-909a-e0120c265f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:33.358030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ad8e-335b-4db5-909a-e0120c265f88 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:33.358312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ad8e-335b-4db5-909a-e0120c265f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:33.358583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ad8e-335b-4db5-909a-e0120c265f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.443073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ad8e-335b-4db5-909a-e0120c265f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:33.444421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ad8e-335b-4db5-909a-e0120c265f88 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:33.548925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 549/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:33.550093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:33.558110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab77333-39c3-4588-b6f5-968fa57cfef6 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:33.558397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab77333-39c3-4588-b6f5-968fa57cfef6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:33.558676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab77333-39c3-4588-b6f5-968fa57cfef6 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.560208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 550/1094] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:29:33.565707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869fc73-6a6e-4b9a-8e50-f980cb9a97e5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:33.565992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869fc73-6a6e-4b9a-8e50-f980cb9a97e5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:33.566271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869fc73-6a6e-4b9a-8e50-f980cb9a97e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.584279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e869fc73-6a6e-4b9a-8e50-f980cb9a97e5 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:33.585521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e869fc73-6a6e-4b9a-8e50-f980cb9a97e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:33.598409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:33.599534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:33.655516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69290f9b-c990-40b5-becd-ccf1530775eb None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:33.664819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 551/1095] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:29:33.690258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:33.691051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:33.691338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:33.691615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.697421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869fc73-6a6e-4b9a-8e50-f980cb9a97e5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:33.702752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:33.704170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:29:33.721618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:33.722630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:33.723511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:29:33.723511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:29:33.723511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:29:33.723511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:33.723511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 545/1096] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:33.728564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:33.817210 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869fc73-6a6e-4b9a-8e50-f980cb9a97e5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:33.823976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:33.829946 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec0f1-65c4-405c-977c-26ce2f2e9c11 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:33.850527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:33.851325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:33.851615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:33.851891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.862401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:33.862747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:33.863407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:33.863696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:33.863973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:33.864720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:33.865028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:33.866402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:33.870041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:33.870956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:33.871784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 552/1097] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:29:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:33.877902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:33.878109 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 546/1098] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:33.883288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:33.888841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:33.977001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f5e49-c6c3-4cc7-b6ed-4775902c94c4 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:33.980740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0476f7f-d748-4734-ae84-947e121e12b3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:34.011517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.012185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.012480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.012789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.013537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:34.014416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.015025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.015782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.015883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.016096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.016377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.022975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.028851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 553/1099] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:29:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:34.028994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.030449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:29:34.039428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:34.041363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:34.042649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 547/1100] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:29:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:29:34.046285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:34.063597 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:34.101699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:34.131999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3e5d0-fc48-4c39-b5df-fb551bd0dd92 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:34.147133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d408445-8354-4ea4-8832-c76c99e86661 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:34.167010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.167876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.168159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.168433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] PATH_INFO: `/v3/users/b551638d1ff242de91e2e4c6f11a8b91` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.169683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] RBAC: Authorizing `identity:get_user(user_id=b551638d1ff242de91e2e4c6f11a8b91)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.171175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.173005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 554/1101] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:34 2022] GET /identity/v3/users/b551638d1ff242de91e2e4c6f11a8b91 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:34.178465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:34.181137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.181799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.182086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.182373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.183165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:34.183480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.184951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.201884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 548/1102] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:34.208071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:34.284616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:34.286046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c89e4-2ec8-493a-a6b8-d7acd0adfab9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:34.306490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1884c7f-636d-4739-9f20-c584a4c1f193 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:34.319709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.320357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.320669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.321155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.330344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:34.341496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.342154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.342496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.342781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.343615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] RBAC: Adding query filter params (role.id=d3f1418897324c3d88c6f01b75cf98e9, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=3ba11eb5837a4567bd77f3fcb0017ea7) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:34.343929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.345439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.351296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 549/1103] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:34 2022] GET /identity/v3/role_assignments?user.id=3ba11eb5837a4567bd77f3fcb0017ea7&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=d3f1418897324c3d88c6f01b75cf98e9 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:34.393955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.395433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.500200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 555/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 17:29:34.506330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:34.620657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a258-4287-4ea8-8d42-148d056c50da None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:34.655112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e62d9-cc39-4365-a11b-07c393e6c087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.655984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e62d9-cc39-4365-a11b-07c393e6c087 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.656264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e62d9-cc39-4365-a11b-07c393e6c087 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.656541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e62d9-cc39-4365-a11b-07c393e6c087 None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.657837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e62d9-cc39-4365-a11b-07c393e6c087 None admin] RBAC: Authorizing `identity:get_project(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.659281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e62d9-cc39-4365-a11b-07c393e6c087 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.661039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 550/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:34 2022] GET /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:34.667488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:34.708332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:34.783579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9038b-3167-40f3-aafe-1dc801f030cc None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:34.820572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ae12f-cced-4a00-a77d-9bf29d81cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:34.821261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ae12f-cced-4a00-a77d-9bf29d81cfdc None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:34.821553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ae12f-cced-4a00-a77d-9bf29d81cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:34.821833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ae12f-cced-4a00-a77d-9bf29d81cfdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:34.822671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ae12f-cced-4a00-a77d-9bf29d81cfdc None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc, user.id=b551638d1ff242de91e2e4c6f11a8b91) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:34.822982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ae12f-cced-4a00-a77d-9bf29d81cfdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:34.824409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ae12f-cced-4a00-a77d-9bf29d81cfdc None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:34.830369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 556/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:34 2022] GET /identity/v3/role_assignments?user.id=b551638d1ff242de91e2e4c6f11a8b91&scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:36.836560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732fadc-f553-40f6-8124-99653bd7fcb7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:36.837166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732fadc-f553-40f6-8124-99653bd7fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:36.837166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732fadc-f553-40f6-8124-99653bd7fcb7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:36.838517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 551/1107] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:36.843595 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c5ba4-c842-4ea3-a7c0-99cd8842f9ad None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:36.844042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c5ba4-c842-4ea3-a7c0-99cd8842f9ad None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:36.844143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c5ba4-c842-4ea3-a7c0-99cd8842f9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:36.860957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8f0c5ba4-c842-4ea3-a7c0-99cd8842f9ad None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:36.862133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8f0c5ba4-c842-4ea3-a7c0-99cd8842f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:36.927435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 557/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:36.951635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732fadc-f553-40f6-8124-99653bd7fcb7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:36.962754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:37.052844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732fadc-f553-40f6-8124-99653bd7fcb7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:37.085191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf6e71-983a-4b50-b954-1a0a79ac8304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:37.085968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf6e71-983a-4b50-b954-1a0a79ac8304 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:37.086252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf6e71-983a-4b50-b954-1a0a79ac8304 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:37.086535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf6e71-983a-4b50-b954-1a0a79ac8304 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:37.087736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf6e71-983a-4b50-b954-1a0a79ac8304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:37.089175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf6e71-983a-4b50-b954-1a0a79ac8304 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:37.090792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 552/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:37.097323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c5ba4-c842-4ea3-a7c0-99cd8842f9ad None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:37.126851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:37.193609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c5ba4-c842-4ea3-a7c0-99cd8842f9ad None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:37.227487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb952ce-b97d-4d12-b256-430d8498066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:37.228161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb952ce-b97d-4d12-b256-430d8498066e None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:37.228448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb952ce-b97d-4d12-b256-430d8498066e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:37.228759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb952ce-b97d-4d12-b256-430d8498066e None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:37.229473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb952ce-b97d-4d12-b256-430d8498066e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:37.230914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb952ce-b97d-4d12-b256-430d8498066e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:37.327312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 558/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:29:37 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:39.383665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7be579-8960-4007-a1e8-31afb0b8c416 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:39.384384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7be579-8960-4007-a1e8-31afb0b8c416 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:39.384384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7be579-8960-4007-a1e8-31afb0b8c416 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:39.385651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 553/1111] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:39.390855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fecf9c-58f6-4cc4-8bcc-d816eb397394 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:39.391264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fecf9c-58f6-4cc4-8bcc-d816eb397394 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:39.391413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fecf9c-58f6-4cc4-8bcc-d816eb397394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:39.408582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f5fecf9c-58f6-4cc4-8bcc-d816eb397394 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:39.409796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f5fecf9c-58f6-4cc4-8bcc-d816eb397394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:39.474687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 559/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:39.497789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7be579-8960-4007-a1e8-31afb0b8c416 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:39.573370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:39.596434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7be579-8960-4007-a1e8-31afb0b8c416 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:39.630232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:39.631078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:39.631369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:39.631652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] PATH_INFO: `/v3/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:39.633226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] RBAC: Authorizing `identity:get_role(role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:39.634606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:39.636011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 554/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:39 2022] GET /identity/v3/roles/d74c967ed68244238636f7d9bb0517c6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:39.642314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fecf9c-58f6-4cc4-8bcc-d816eb397394 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:39.661008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:39.740870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fecf9c-58f6-4cc4-8bcc-d816eb397394 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:39.775680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:39.775680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:39.775680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:39.776036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] PATH_INFO: `/v3/users/0eae81e2d29645b6974cc15d36d8e57f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:39.789224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] RBAC: Authorizing `identity:get_user(user_id=0eae81e2d29645b6974cc15d36d8e57f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:39.790661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:39.792313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 560/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:39 2022] GET /identity/v3/users/0eae81e2d29645b6974cc15d36d8e57f => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:39.798187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:39.893674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c9fc-5f35-4bd5-96be-2657129622df None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:39.927469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:39.928105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:39.928388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:39.928689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:39.942661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:39.993568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:39.995030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:40.090309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 555/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:40.096797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:40.171824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:40.193963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbc0d1-86a8-411a-b007-3c900d7ef3cb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:40.227629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ac095-6987-4e4c-a86f-072ee99ca793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:40.228494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac095-6987-4e4c-a86f-072ee99ca793 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:40.228809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac095-6987-4e4c-a86f-072ee99ca793 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:40.229089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac095-6987-4e4c-a86f-072ee99ca793 None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:40.230282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ac095-6987-4e4c-a86f-072ee99ca793 None admin] RBAC: Authorizing `identity:get_project(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:40.231706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ac095-6987-4e4c-a86f-072ee99ca793 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:40.234058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 561/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:40 2022] GET /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:40.240104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:40.284200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:40.336084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb78ce-91cc-4375-a085-43707873581c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:40.369486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a144d91-94b4-4dc2-8da2-821dbeeb00e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:40.370069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144d91-94b4-4dc2-8da2-821dbeeb00e0 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:40.370359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144d91-94b4-4dc2-8da2-821dbeeb00e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:40.370641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144d91-94b4-4dc2-8da2-821dbeeb00e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:40.371476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144d91-94b4-4dc2-8da2-821dbeeb00e0 None admin] RBAC: Adding query filter params (role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc, user.id=0eae81e2d29645b6974cc15d36d8e57f) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:40.371793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144d91-94b4-4dc2-8da2-821dbeeb00e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:40.373327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144d91-94b4-4dc2-8da2-821dbeeb00e0 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:40.378632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 556/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:40 2022] GET /identity/v3/role_assignments?user.id=0eae81e2d29645b6974cc15d36d8e57f&scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:42.332297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adeb2f-0d39-4025-94cb-3f949dfb1e77 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:42.332861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adeb2f-0d39-4025-94cb-3f949dfb1e77 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:42.332955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adeb2f-0d39-4025-94cb-3f949dfb1e77 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:42.334281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 562/1118] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:42.339615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce4bb-c8e7-405a-9c39-1c290416c548 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:42.340010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce4bb-c8e7-405a-9c39-1c290416c548 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:42.340170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ce4bb-c8e7-405a-9c39-1c290416c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:42.357050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3f8ce4bb-c8e7-405a-9c39-1c290416c548 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:42.359003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3f8ce4bb-c8e7-405a-9c39-1c290416c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:42.423789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 557/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:42.445627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adeb2f-0d39-4025-94cb-3f949dfb1e77 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:42.544642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adeb2f-0d39-4025-94cb-3f949dfb1e77 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:42.576994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:42.577840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:42.578123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:42.578397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] PATH_INFO: `/v3/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:42.579860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] RBAC: Authorizing `identity:get_role(role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:42.581251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:42.589816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 563/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:42 2022] GET /identity/v3/roles/d74c967ed68244238636f7d9bb0517c6 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:42.596139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce4bb-c8e7-405a-9c39-1c290416c548 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:42.634351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:42.699345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ce4bb-c8e7-405a-9c39-1c290416c548 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:42.733287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:42.734148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:42.734436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:42.734714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] PATH_INFO: `/v3/users/0eae81e2d29645b6974cc15d36d8e57f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:42.735887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] RBAC: Authorizing `identity:get_user(user_id=0eae81e2d29645b6974cc15d36d8e57f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:42.737356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:42.738988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 558/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:42 2022] GET /identity/v3/users/0eae81e2d29645b6974cc15d36d8e57f => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:42.744997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:42.759557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:42.842816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831b264-7a45-4611-a888-4da18d86ee40 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:42.875392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3838f2-e62c-4ef9-9cb1-b548e4db3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:42.876244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3838f2-e62c-4ef9-9cb1-b548e4db3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:42.876533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3838f2-e62c-4ef9-9cb1-b548e4db3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:42.876841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3838f2-e62c-4ef9-9cb1-b548e4db3fe6 None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:42.878016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3838f2-e62c-4ef9-9cb1-b548e4db3fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:42.879436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3838f2-e62c-4ef9-9cb1-b548e4db3fe6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:42.881156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 564/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:42 2022] GET /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:42.886988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:42.986837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09174cdc-1d9d-47f2-b28c-65682e3b34b1 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:42.994715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:43.020785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa68ba-4335-4a10-9264-5d5c240bd984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:43.021680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa68ba-4335-4a10-9264-5d5c240bd984 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:43.021962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa68ba-4335-4a10-9264-5d5c240bd984 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:43.022244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa68ba-4335-4a10-9264-5d5c240bd984 None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc/users/0eae81e2d29645b6974cc15d36d8e57f/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:43.024770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa68ba-4335-4a10-9264-5d5c240bd984 None admin] RBAC: Authorizing `identity:create_grant(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc, user_id=0eae81e2d29645b6974cc15d36d8e57f, role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:43.026210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa68ba-4335-4a10-9264-5d5c240bd984 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:43.127250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 559/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:29:42 2022] PUT /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc/users/0eae81e2d29645b6974cc15d36d8e57f/roles/d74c967ed68244238636f7d9bb0517c6 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:29:45.108206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcffc2-35a4-41cc-8b25-2a44dc11ae1f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:45.108788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcffc2-35a4-41cc-8b25-2a44dc11ae1f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:45.108788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcffc2-35a4-41cc-8b25-2a44dc11ae1f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:45.110175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 565/1124] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:45.115488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1715ef-2b88-40ff-9f1b-c5e87e14e68d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:45.115858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1715ef-2b88-40ff-9f1b-c5e87e14e68d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:45.116051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1715ef-2b88-40ff-9f1b-c5e87e14e68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:45.133625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ff1715ef-2b88-40ff-9f1b-c5e87e14e68d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:45.134810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ff1715ef-2b88-40ff-9f1b-c5e87e14e68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:45.200454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 560/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:45.223461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcffc2-35a4-41cc-8b25-2a44dc11ae1f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:45.318834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:45.338200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcffc2-35a4-41cc-8b25-2a44dc11ae1f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:45.372896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:45.373682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:45.373966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:45.374244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] PATH_INFO: `/v3/roles/d74c967ed68244238636f7d9bb0517c6` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:45.375707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] RBAC: Authorizing `identity:get_role(role_id=d74c967ed68244238636f7d9bb0517c6)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:45.377111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:45.378496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 566/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:45 2022] GET /identity/v3/roles/d74c967ed68244238636f7d9bb0517c6 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:45.384710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1715ef-2b88-40ff-9f1b-c5e87e14e68d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:45.475266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:45.481095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1715ef-2b88-40ff-9f1b-c5e87e14e68d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:45.514545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:45.515401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:45.515684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:45.515960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] PATH_INFO: `/v3/users/0eae81e2d29645b6974cc15d36d8e57f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:45.517158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] RBAC: Authorizing `identity:get_user(user_id=0eae81e2d29645b6974cc15d36d8e57f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:45.518590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:45.520194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 561/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:45 2022] GET /identity/v3/users/0eae81e2d29645b6974cc15d36d8e57f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:45.526253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:45.621069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b3356-b25a-4daa-8abb-cf48bc475205 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:45.658590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:45.659602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:45.660099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:45.660619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:45.681244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:45.729965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:45.731399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:45.825203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 567/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:45.831697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:45.928504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122f92e-e7e6-416c-82df-333c26667277 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:45.966296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5411fe-2dd9-49f5-9c03-2021c868afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:45.967728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5411fe-2dd9-49f5-9c03-2021c868afdb None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:45.968017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5411fe-2dd9-49f5-9c03-2021c868afdb None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:45.968298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5411fe-2dd9-49f5-9c03-2021c868afdb None admin] PATH_INFO: `/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:45.969586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5411fe-2dd9-49f5-9c03-2021c868afdb None admin] RBAC: Authorizing `identity:get_project(project_id=71fff8f0fc1043f18e9b2bc2b0a1aecc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:45.971038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5411fe-2dd9-49f5-9c03-2021c868afdb None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:45.972779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 562/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:45 2022] GET /identity/v3/projects/71fff8f0fc1043f18e9b2bc2b0a1aecc => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:45.978975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:46.026830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:46.075771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0684b-f8f7-42f6-8cbe-207a043e42bf None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:46.115098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421efdfd-6a24-4415-8b77-e1043fe0313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:46.115739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421efdfd-6a24-4415-8b77-e1043fe0313f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:46.116024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421efdfd-6a24-4415-8b77-e1043fe0313f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:46.116308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421efdfd-6a24-4415-8b77-e1043fe0313f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:46.117174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421efdfd-6a24-4415-8b77-e1043fe0313f None admin] RBAC: Adding query filter params (role.id=d74c967ed68244238636f7d9bb0517c6, scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc, user.id=0eae81e2d29645b6974cc15d36d8e57f) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:46.117493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421efdfd-6a24-4415-8b77-e1043fe0313f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:46.118920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421efdfd-6a24-4415-8b77-e1043fe0313f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:46.125995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 568/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:45 2022] GET /identity/v3/role_assignments?user.id=0eae81e2d29645b6974cc15d36d8e57f&scope.project.id=71fff8f0fc1043f18e9b2bc2b0a1aecc&role.id=d74c967ed68244238636f7d9bb0517c6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:48.143117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90853e-e964-4bfc-a3a3-536a4d7aa91e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:48.143645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90853e-e964-4bfc-a3a3-536a4d7aa91e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:48.143645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90853e-e964-4bfc-a3a3-536a4d7aa91e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:48.145097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 563/1131] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:48.150588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd90637-2985-4a3d-ab77-71a2de60bf7d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:48.150990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd90637-2985-4a3d-ab77-71a2de60bf7d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:48.151150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd90637-2985-4a3d-ab77-71a2de60bf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:48.168113 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3fd90637-2985-4a3d-ab77-71a2de60bf7d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:48.169333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3fd90637-2985-4a3d-ab77-71a2de60bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:48.239061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 569/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:48.261620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90853e-e964-4bfc-a3a3-536a4d7aa91e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:48.293089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:48.361411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90853e-e964-4bfc-a3a3-536a4d7aa91e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:48.394807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065f9fc-7ba7-4a44-b097-87a1ed260a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:48.395523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065f9fc-7ba7-4a44-b097-87a1ed260a6c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:48.395811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065f9fc-7ba7-4a44-b097-87a1ed260a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:48.396091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065f9fc-7ba7-4a44-b097-87a1ed260a6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:48.397291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065f9fc-7ba7-4a44-b097-87a1ed260a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:48.398739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065f9fc-7ba7-4a44-b097-87a1ed260a6c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:48.400634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 564/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:48.406660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd90637-2985-4a3d-ab77-71a2de60bf7d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:48.448315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:48.502396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd90637-2985-4a3d-ab77-71a2de60bf7d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:48.535865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f1e64-1764-43ef-8f1d-ac2a59c396d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:48.536506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f1e64-1764-43ef-8f1d-ac2a59c396d9 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:48.536824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f1e64-1764-43ef-8f1d-ac2a59c396d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:48.537100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f1e64-1764-43ef-8f1d-ac2a59c396d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:48.537806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f1e64-1764-43ef-8f1d-ac2a59c396d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:48.539317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f1e64-1764-43ef-8f1d-ac2a59c396d9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:48.647860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 570/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:29:48 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:50.661253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627daca7-11d5-48a3-ba3c-74d145d9699b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:50.661787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627daca7-11d5-48a3-ba3c-74d145d9699b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:50.661908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627daca7-11d5-48a3-ba3c-74d145d9699b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:50.663219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 565/1135] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:50.668478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845dd20-6ace-40bf-814a-efd8897f65ba None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:50.668944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845dd20-6ace-40bf-814a-efd8897f65ba None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:50.669075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845dd20-6ace-40bf-814a-efd8897f65ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:50.686178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1845dd20-6ace-40bf-814a-efd8897f65ba None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:50.687369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1845dd20-6ace-40bf-814a-efd8897f65ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:50.751596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 571/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:50.776208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627daca7-11d5-48a3-ba3c-74d145d9699b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:50.870684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:50.875129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627daca7-11d5-48a3-ba3c-74d145d9699b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:50.907769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11371247-798e-4faa-9d62-2e709e860a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:50.908557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11371247-798e-4faa-9d62-2e709e860a84 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:50.909065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11371247-798e-4faa-9d62-2e709e860a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:50.909570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11371247-798e-4faa-9d62-2e709e860a84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:50.911032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11371247-798e-4faa-9d62-2e709e860a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:50.912541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11371247-798e-4faa-9d62-2e709e860a84 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:50.914147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 566/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:29:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:50.919965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845dd20-6ace-40bf-814a-efd8897f65ba None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:50.944190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:51.018104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845dd20-6ace-40bf-814a-efd8897f65ba None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:51.051103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fe6c6-1bb7-4f8c-a27a-2ca606d10907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:51.051767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe6c6-1bb7-4f8c-a27a-2ca606d10907 None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:51.052059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe6c6-1bb7-4f8c-a27a-2ca606d10907 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:51.052340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe6c6-1bb7-4f8c-a27a-2ca606d10907 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:51.053085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fe6c6-1bb7-4f8c-a27a-2ca606d10907 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:51.054520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fe6c6-1bb7-4f8c-a27a-2ca606d10907 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:51.155332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 572/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:29:50 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:29:53.182416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b64da-2706-4473-9378-9b57b582d837 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:53.182971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b64da-2706-4473-9378-9b57b582d837 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:53.182971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b64da-2706-4473-9378-9b57b582d837 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:53.184361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 567/1139] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:53.189980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71cf0-2111-438a-bf9a-5ccadd3c4ef6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:53.190397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71cf0-2111-438a-bf9a-5ccadd3c4ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:53.190541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71cf0-2111-438a-bf9a-5ccadd3c4ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:53.207636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6aa71cf0-2111-438a-bf9a-5ccadd3c4ef6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:53.208852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6aa71cf0-2111-438a-bf9a-5ccadd3c4ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:53.274839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 573/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:53.298023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b64da-2706-4473-9378-9b57b582d837 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:53.397958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b64da-2706-4473-9378-9b57b582d837 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:53.430558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:53.431308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:53.431593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:53.431869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:53.442412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:53.449750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:53.450958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:53.452087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 568/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:29:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:53.457165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa71cf0-2111-438a-bf9a-5ccadd3c4ef6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:53.469279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:53.563894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa71cf0-2111-438a-bf9a-5ccadd3c4ef6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:53.599357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:53.600033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:53.600319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:53.600633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:53.601379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:53.601684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:53.603068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:53.612301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 574/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:53.618121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:53.646605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:53.715408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9b223-cbec-45fd-b42c-d331c8ab73a3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:53.747812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:53.748718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:53.749008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:53.749284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] PATH_INFO: `/v3/users/6fc263693eb942df8cba2de39f26ffa1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:53.761858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] RBAC: Authorizing `identity:get_user(user_id=6fc263693eb942df8cba2de39f26ffa1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:53.763350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:53.765017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 569/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:53 2022] GET /identity/v3/users/6fc263693eb942df8cba2de39f26ffa1 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:53.770777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:53.838341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:53.867251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdebaf-2dd0-4324-b054-ac473d589178 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:53.899655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:53.900287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:53.900570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:53.900875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:53.965567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:53.966954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:54.032738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:54.062050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 575/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:29:54.067921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:54.148977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:54.163652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591f124-8162-4cd9-a8ae-5008f789c617 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:54.197270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabaf35-28d7-4fa5-a475-41a73561ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:54.198111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabaf35-28d7-4fa5-a475-41a73561ff36 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:54.198395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabaf35-28d7-4fa5-a475-41a73561ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:54.198668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabaf35-28d7-4fa5-a475-41a73561ff36 None admin] PATH_INFO: `/v3/projects/55b40a04d9c843d1b1fe15455b5712dc` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:54.199814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabaf35-28d7-4fa5-a475-41a73561ff36 None admin] RBAC: Authorizing `identity:get_project(project_id=55b40a04d9c843d1b1fe15455b5712dc)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:54.201294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabaf35-28d7-4fa5-a475-41a73561ff36 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:54.202961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 570/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:54 2022] GET /identity/v3/projects/55b40a04d9c843d1b1fe15455b5712dc => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:54.209048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:54.305860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e7303-a8bb-41bb-8d45-b0abc6687860 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:54.339184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10676747-1574-4518-96fc-c5ece787f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:54.339829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676747-1574-4518-96fc-c5ece787f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:54.340116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676747-1574-4518-96fc-c5ece787f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:54.340396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676747-1574-4518-96fc-c5ece787f2e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:54.341781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10676747-1574-4518-96fc-c5ece787f2e0 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=55b40a04d9c843d1b1fe15455b5712dc, user.id=6fc263693eb942df8cba2de39f26ffa1) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:54.342094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10676747-1574-4518-96fc-c5ece787f2e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:54.343531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10676747-1574-4518-96fc-c5ece787f2e0 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:54.352137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 576/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:29:54 2022] GET /identity/v3/role_assignments?user.id=6fc263693eb942df8cba2de39f26ffa1&scope.project.id=55b40a04d9c843d1b1fe15455b5712dc&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:56.328744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2f3fd-740f-42a2-91f9-2ee3b21b68ed None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:56.329242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2f3fd-740f-42a2-91f9-2ee3b21b68ed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:56.329334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2f3fd-740f-42a2-91f9-2ee3b21b68ed None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:56.330679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 571/1147] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:56.335804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553bf99-dd47-431d-9db8-1fa7999bb0ce None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:56.336181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553bf99-dd47-431d-9db8-1fa7999bb0ce None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:56.336374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553bf99-dd47-431d-9db8-1fa7999bb0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:56.352808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f553bf99-dd47-431d-9db8-1fa7999bb0ce None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:56.353994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f553bf99-dd47-431d-9db8-1fa7999bb0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:56.382833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:56.419918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 577/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:56.442660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2f3fd-740f-42a2-91f9-2ee3b21b68ed None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:56.539713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2f3fd-740f-42a2-91f9-2ee3b21b68ed None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:56.572503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:56.573324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:56.573610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:56.573887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:56.583746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:56.591753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:56.592904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:29:56.593978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 572/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:29:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:56.599126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553bf99-dd47-431d-9db8-1fa7999bb0ce None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:56.695301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553bf99-dd47-431d-9db8-1fa7999bb0ce None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:56.737198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:56.737869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:56.738156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:56.738435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:56.739187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:56.739494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:56.740883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:56.749072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 578/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:29:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:56.754928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:56.770805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:56.853844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821dbf2-9c17-4a2c-8306-59be31e15af2 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:56.886589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:56.887438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:56.887727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:56.888043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] PATH_INFO: `/v3/users/6fc263693eb942df8cba2de39f26ffa1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:56.889232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] RBAC: Authorizing `identity:get_user(user_id=6fc263693eb942df8cba2de39f26ffa1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:56.890714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:56.892331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 573/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:56 2022] GET /identity/v3/users/6fc263693eb942df8cba2de39f26ffa1 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:56.898016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:56.927019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:56.995563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475861d-33ae-4f4b-b528-5519a70bbed6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:57.037513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee00e83-d1a2-4bcf-8a50-f2ce4a42819c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:57.038385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee00e83-d1a2-4bcf-8a50-f2ce4a42819c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:57.038671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee00e83-d1a2-4bcf-8a50-f2ce4a42819c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:57.038945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee00e83-d1a2-4bcf-8a50-f2ce4a42819c None admin] PATH_INFO: `/v3/projects/55b40a04d9c843d1b1fe15455b5712dc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:57.040129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee00e83-d1a2-4bcf-8a50-f2ce4a42819c None admin] RBAC: Authorizing `identity:get_project(project_id=55b40a04d9c843d1b1fe15455b5712dc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:57.041609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee00e83-d1a2-4bcf-8a50-f2ce4a42819c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:57.043347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 579/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:29:56 2022] GET /identity/v3/projects/55b40a04d9c843d1b1fe15455b5712dc => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:57.049408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:57.127201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:57.145212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd12fd-d869-4ff5-bc34-52c1b03389ea None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:57.178496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65112862-930d-4d1c-af18-5b6c2f1e4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:57.179367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65112862-930d-4d1c-af18-5b6c2f1e4280 None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:57.179653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65112862-930d-4d1c-af18-5b6c2f1e4280 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:57.179928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65112862-930d-4d1c-af18-5b6c2f1e4280 None admin] PATH_INFO: `/v3/projects/55b40a04d9c843d1b1fe15455b5712dc/users/6fc263693eb942df8cba2de39f26ffa1/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:57.181861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65112862-930d-4d1c-af18-5b6c2f1e4280 None admin] RBAC: Authorizing `identity:create_grant(project_id=55b40a04d9c843d1b1fe15455b5712dc, user_id=6fc263693eb942df8cba2de39f26ffa1, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:57.183295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65112862-930d-4d1c-af18-5b6c2f1e4280 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:57.272103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 574/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:29:57 2022] PUT /identity/v3/projects/55b40a04d9c843d1b1fe15455b5712dc/users/6fc263693eb942df8cba2de39f26ffa1/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:29:59.258636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02b741-dbee-4749-8156-b41500cffe3c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:59.259122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02b741-dbee-4749-8156-b41500cffe3c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:59.259216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02b741-dbee-4749-8156-b41500cffe3c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:59.260562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 580/1154] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:29:59.265599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12798e-a934-4a2f-96d7-570d3352fb2c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:59.265995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12798e-a934-4a2f-96d7-570d3352fb2c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:59.266148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12798e-a934-4a2f-96d7-570d3352fb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:59.283237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6c12798e-a934-4a2f-96d7-570d3352fb2c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:29:59.284406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6c12798e-a934-4a2f-96d7-570d3352fb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:29:59.347760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 575/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:29:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:29:59.372754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02b741-dbee-4749-8156-b41500cffe3c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:59.466325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:59.473032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02b741-dbee-4749-8156-b41500cffe3c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:59.506701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:59.507417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:59.507708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:59.507988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:59.518448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:29:59.526291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:29:59.527376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:29:59.528434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 581/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:29:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:29:59.533641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12798e-a934-4a2f-96d7-570d3352fb2c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:59.599594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:29:59.631607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12798e-a934-4a2f-96d7-570d3352fb2c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:59.665030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:59.665691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:59.665980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:59.666260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:59.667014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:29:59.667325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:59.668744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:59.677620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 576/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:29:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:29:59.682962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:59.780480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b040e4-6cf5-40f4-85b9-9964bd5db2b6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:59.812786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:59.813643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:59.813931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:59.814211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] PATH_INFO: `/v3/users/6fc263693eb942df8cba2de39f26ffa1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:59.815386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] RBAC: Authorizing `identity:get_user(user_id=6fc263693eb942df8cba2de39f26ffa1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:29:59.816846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:29:59.818462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 582/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:29:59 2022] GET /identity/v3/users/6fc263693eb942df8cba2de39f26ffa1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:29:59.824193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:29:59.921193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21081d6-a348-4fb8-b5e5-2d17ff3f3cd8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:29:59.953468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:29:59.954098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:29:59.954383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:29:59.954657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:29:59.957196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:00.019516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:00.020997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:00.115245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 577/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:29:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:30:00.121163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:00.146610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:00.218777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d87f8-1078-4076-9e01-b79fb9235d1b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:00.253861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11247c50-a210-4928-9b1b-cb575763f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:00.254719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247c50-a210-4928-9b1b-cb575763f00b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:00.255005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247c50-a210-4928-9b1b-cb575763f00b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:00.255284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247c50-a210-4928-9b1b-cb575763f00b None admin] PATH_INFO: `/v3/projects/55b40a04d9c843d1b1fe15455b5712dc` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:00.256458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11247c50-a210-4928-9b1b-cb575763f00b None admin] RBAC: Authorizing `identity:get_project(project_id=55b40a04d9c843d1b1fe15455b5712dc)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:00.257926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11247c50-a210-4928-9b1b-cb575763f00b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:00.259633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 583/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:30:00 2022] GET /identity/v3/projects/55b40a04d9c843d1b1fe15455b5712dc => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:00.265820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:00.299171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:00.362891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7340a1-1f49-41f9-9e5d-db0fc62dc590 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:00.396475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef9f2f-9461-4205-a4cf-72453d8befb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:00.397075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef9f2f-9461-4205-a4cf-72453d8befb8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:00.397359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef9f2f-9461-4205-a4cf-72453d8befb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:00.397631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef9f2f-9461-4205-a4cf-72453d8befb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:00.398537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef9f2f-9461-4205-a4cf-72453d8befb8 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=55b40a04d9c843d1b1fe15455b5712dc, user.id=6fc263693eb942df8cba2de39f26ffa1) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:30:00.398847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef9f2f-9461-4205-a4cf-72453d8befb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:00.400628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef9f2f-9461-4205-a4cf-72453d8befb8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:00.406777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 578/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:30:00 2022] GET /identity/v3/role_assignments?user.id=6fc263693eb942df8cba2de39f26ffa1&scope.project.id=55b40a04d9c843d1b1fe15455b5712dc&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:02.441870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d4c4-1a47-4683-87af-db18e14bc9a7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:02.442391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d4c4-1a47-4683-87af-db18e14bc9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:02.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d4c4-1a47-4683-87af-db18e14bc9a7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:02.443816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 584/1162] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:30:02.448788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab092c0-10b5-4326-9a58-9670be9a29df None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:02.449171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab092c0-10b5-4326-9a58-9670be9a29df None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:02.449355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab092c0-10b5-4326-9a58-9670be9a29df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:02.466787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5ab092c0-10b5-4326-9a58-9670be9a29df None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:30:02.467974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5ab092c0-10b5-4326-9a58-9670be9a29df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:30:02.533277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 579/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:30:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:30:02.555006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054d4c4-1a47-4683-87af-db18e14bc9a7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:02.653225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:02.662568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054d4c4-1a47-4683-87af-db18e14bc9a7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:02.695945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:02.696588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:02.696904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:02.697185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:02.764522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:02.766003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:02.851737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:02.860865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 585/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:30:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:30:02.866880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab092c0-10b5-4326-9a58-9670be9a29df None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:02.896375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:02.967009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab092c0-10b5-4326-9a58-9670be9a29df None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:03.000199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c8192-875a-4bff-8f3a-6d90d5e95f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:03.001023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c8192-875a-4bff-8f3a-6d90d5e95f74 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:03.001312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c8192-875a-4bff-8f3a-6d90d5e95f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:03.001591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c8192-875a-4bff-8f3a-6d90d5e95f74 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:03.008097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c8192-875a-4bff-8f3a-6d90d5e95f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:03.009581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c8192-875a-4bff-8f3a-6d90d5e95f74 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-107c8192-875a-4bff-8f3a-6d90d5e95f74 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:30:03.023561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:30:03.025082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:30:03.026221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 580/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:30:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:30:03.031306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:03.148682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d90800-2e7a-42f8-b9fe-1752fc1a39e2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:03.182013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bf589-c3b7-4d7d-94d4-f4ac96b36a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:03.182013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bf589-c3b7-4d7d-94d4-f4ac96b36a66 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:03.182296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bf589-c3b7-4d7d-94d4-f4ac96b36a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:03.182374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bf589-c3b7-4d7d-94d4-f4ac96b36a66 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:03.183070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bf589-c3b7-4d7d-94d4-f4ac96b36a66 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:30:03.183386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bf589-c3b7-4d7d-94d4-f4ac96b36a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:03.185361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bf589-c3b7-4d7d-94d4-f4ac96b36a66 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:03.194548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 586/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Jul 15 17:30:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:05.202461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d5f64-b85a-4178-97d4-ae8dab524288 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:05.203098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d5f64-b85a-4178-97d4-ae8dab524288 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:05.203098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d5f64-b85a-4178-97d4-ae8dab524288 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:05.204442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 581/1167] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:30:05.209755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23d7c2-8e50-4899-bf0b-a45614052c67 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:05.210207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23d7c2-8e50-4899-bf0b-a45614052c67 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:05.210343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23d7c2-8e50-4899-bf0b-a45614052c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:05.227790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ed23d7c2-8e50-4899-bf0b-a45614052c67 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:30:05.229024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ed23d7c2-8e50-4899-bf0b-a45614052c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:30:05.279794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:05.296386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 587/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:30:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:30:05.319839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d5f64-b85a-4178-97d4-ae8dab524288 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:05.407850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:05.420629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d5f64-b85a-4178-97d4-ae8dab524288 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:05.453943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ecffa5-408d-4454-9bc3-6c2719ed2c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:05.454536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecffa5-408d-4454-9bc3-6c2719ed2c87 None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:05.454823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecffa5-408d-4454-9bc3-6c2719ed2c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:05.455105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecffa5-408d-4454-9bc3-6c2719ed2c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:05.455848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecffa5-408d-4454-9bc3-6c2719ed2c87 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:05.457278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecffa5-408d-4454-9bc3-6c2719ed2c87 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:05.557862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 582/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Fri Jul 15 17:30:05 2022] POST /identity/v3/domains => generated 267 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:30:07.566439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca00b2-3910-49e8-9dc3-8af2bee92cc7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:07.566988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca00b2-3910-49e8-9dc3-8af2bee92cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:07.566988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca00b2-3910-49e8-9dc3-8af2bee92cc7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:07.568389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 588/1170] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:30:07.573692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772b24-5cf3-4ab6-bbf1-f08fa0591e3b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:07.574110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772b24-5cf3-4ab6-bbf1-f08fa0591e3b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:07.574255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772b24-5cf3-4ab6-bbf1-f08fa0591e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:07.591192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-92772b24-5cf3-4ab6-bbf1-f08fa0591e3b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:30:07.592416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-92772b24-5cf3-4ab6-bbf1-f08fa0591e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:30:07.659201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 583/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:30:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:30:07.683433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca00b2-3910-49e8-9dc3-8af2bee92cc7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:07.784346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca00b2-3910-49e8-9dc3-8af2bee92cc7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:07.817987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134645c-5537-4a4f-b206-fe4bcda1ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:07.818776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134645c-5537-4a4f-b206-fe4bcda1ef2e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:07.819064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134645c-5537-4a4f-b206-fe4bcda1ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:07.819343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134645c-5537-4a4f-b206-fe4bcda1ef2e None admin] PATH_INFO: `/v3/domains/5d23b8a93031467f8fc6d575166e826a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:07.829662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134645c-5537-4a4f-b206-fe4bcda1ef2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=5d23b8a93031467f8fc6d575166e826a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:07.831112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134645c-5537-4a4f-b206-fe4bcda1ef2e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:07.832789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 589/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Fri Jul 15 17:30:07 2022] GET /identity/v3/domains/5d23b8a93031467f8fc6d575166e826a => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:07.839105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92772b24-5cf3-4ab6-bbf1-f08fa0591e3b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:07.915490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:07.936462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92772b24-5cf3-4ab6-bbf1-f08fa0591e3b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:07.969831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27109a79-fed1-4c1f-9d04-44a59fbca57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:07.970527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109a79-fed1-4c1f-9d04-44a59fbca57c None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:07.970823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109a79-fed1-4c1f-9d04-44a59fbca57c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:07.971298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27109a79-fed1-4c1f-9d04-44a59fbca57c None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:07.972183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109a79-fed1-4c1f-9d04-44a59fbca57c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:07.974117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27109a79-fed1-4c1f-9d04-44a59fbca57c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:08.054446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 584/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:30:07 2022] POST /identity/v3/projects => generated 361 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:30:10.102259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeac87e-efcb-4252-b2c1-839ce69def84 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:10.102797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeac87e-efcb-4252-b2c1-839ce69def84 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:10.102900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeac87e-efcb-4252-b2c1-839ce69def84 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:10.104208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 590/1174] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:30:10.109993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06521-6d45-4547-ac87-d78ed126f0e8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:10.110408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06521-6d45-4547-ac87-d78ed126f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:10.110555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06521-6d45-4547-ac87-d78ed126f0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:10.126998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-adc06521-6d45-4547-ac87-d78ed126f0e8 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:30:10.128172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-adc06521-6d45-4547-ac87-d78ed126f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:30:10.191925 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 585/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:30:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:30:10.215626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeac87e-efcb-4252-b2c1-839ce69def84 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:10.263444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:10.317368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeac87e-efcb-4252-b2c1-839ce69def84 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:10.350307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be551812-28e9-4374-b65d-a215567cad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:10.351167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551812-28e9-4374-b65d-a215567cad3a None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:10.351450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551812-28e9-4374-b65d-a215567cad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:10.351727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551812-28e9-4374-b65d-a215567cad3a None admin] PATH_INFO: `/v3/domains/5d23b8a93031467f8fc6d575166e826a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:10.352986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551812-28e9-4374-b65d-a215567cad3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=5d23b8a93031467f8fc6d575166e826a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:10.354386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551812-28e9-4374-b65d-a215567cad3a None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:10.356020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 591/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Fri Jul 15 17:30:10 2022] GET /identity/v3/domains/5d23b8a93031467f8fc6d575166e826a => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:10.362505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06521-6d45-4547-ac87-d78ed126f0e8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:10.421475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:10.460004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06521-6d45-4547-ac87-d78ed126f0e8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:10.492968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068e9a8-71d9-4d5d-a201-a4e9f9f2344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:10.493634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e9a8-71d9-4d5d-a201-a4e9f9f2344d None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:10.493924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e9a8-71d9-4d5d-a201-a4e9f9f2344d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:10.494205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e9a8-71d9-4d5d-a201-a4e9f9f2344d None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:10.494914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e9a8-71d9-4d5d-a201-a4e9f9f2344d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:10.496403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e9a8-71d9-4d5d-a201-a4e9f9f2344d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:10.600541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 586/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:30:10 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:30:12.632704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd031af-e921-4a35-928b-625ffd852495 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:12.633251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd031af-e921-4a35-928b-625ffd852495 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:12.633251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd031af-e921-4a35-928b-625ffd852495 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:12.634651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 592/1178] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:30:12.640022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8014e68-a557-462d-abc5-9b32cd28ea88 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:12.640451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8014e68-a557-462d-abc5-9b32cd28ea88 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:12.640585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8014e68-a557-462d-abc5-9b32cd28ea88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:12.657761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c8014e68-a557-462d-abc5-9b32cd28ea88 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:30:12.658950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c8014e68-a557-462d-abc5-9b32cd28ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:30:12.726052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 587/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:30:12.761634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd031af-e921-4a35-928b-625ffd852495 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:12.869895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd031af-e921-4a35-928b-625ffd852495 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:12.886184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:12.905039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:12.906683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:12.906683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:12.906683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:12.916877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:30:12.924578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:30:12.925765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:30:12.926843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 593/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:30:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:30:12.931896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8014e68-a557-462d-abc5-9b32cd28ea88 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:12.978742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:13.030483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8014e68-a557-462d-abc5-9b32cd28ea88 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:13.064159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:13.064855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:13.065146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:13.065428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:13.066183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:30:13.066495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:13.067890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:13.076906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 588/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:30:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:13.082738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:13.181111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb101aa-f951-4328-ae37-395ed00b5892 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:13.213558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:13.214417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:13.214698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:13.214971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] PATH_INFO: `/v3/users/ddf1a6c64532436a87861e208986bd4f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:13.228424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf1a6c64532436a87861e208986bd4f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:13.229902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:13.231665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 594/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:30:13 2022] GET /identity/v3/users/ddf1a6c64532436a87861e208986bd4f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:13.238171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:13.334171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b61b5e-1eed-443f-a960-6e208829f3d6 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:13.343283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:13.367339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:13.368063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:13.368344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:13.368638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:13.434832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:13.436234 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:13.532816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 589/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:30:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:30:13.538689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:13.570838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:13.646220 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e17a8f-c65d-463c-bd23-1346e8195840 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:13.680265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf377556-6041-4100-9d1b-8f62663294ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:13.681158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf377556-6041-4100-9d1b-8f62663294ea None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:13.681445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf377556-6041-4100-9d1b-8f62663294ea None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:13.681725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf377556-6041-4100-9d1b-8f62663294ea None admin] PATH_INFO: `/v3/projects/e0f36ad4b70149bcb304dc804d37b88f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:13.682909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf377556-6041-4100-9d1b-8f62663294ea None admin] RBAC: Authorizing `identity:get_project(project_id=e0f36ad4b70149bcb304dc804d37b88f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:13.684338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf377556-6041-4100-9d1b-8f62663294ea None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:13.686135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 595/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:30:13 2022] GET /identity/v3/projects/e0f36ad4b70149bcb304dc804d37b88f => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:13.692216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:13.703817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:13.794722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35b4e6-550d-41e4-b597-36294192e71e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:13.831144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7385488-c13c-464b-b7ca-9e671f538d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:13.831802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7385488-c13c-464b-b7ca-9e671f538d8b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:13.832089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7385488-c13c-464b-b7ca-9e671f538d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:13.832370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7385488-c13c-464b-b7ca-9e671f538d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:13.833247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7385488-c13c-464b-b7ca-9e671f538d8b None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=e0f36ad4b70149bcb304dc804d37b88f, user.id=ddf1a6c64532436a87861e208986bd4f) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:30:13.833566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7385488-c13c-464b-b7ca-9e671f538d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:13.835002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7385488-c13c-464b-b7ca-9e671f538d8b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:13.840377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 590/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:30:13 2022] GET /identity/v3/role_assignments?user.id=ddf1a6c64532436a87861e208986bd4f&scope.project.id=e0f36ad4b70149bcb304dc804d37b88f&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:15.811265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c65151-7371-43be-a74f-82c9f92d923c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:15.811783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c65151-7371-43be-a74f-82c9f92d923c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:15.811901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c65151-7371-43be-a74f-82c9f92d923c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:15.813225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 596/1186] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:30:15.818320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee2a58-4ba2-4b5b-9831-4c41b85d2f65 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:15.818755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee2a58-4ba2-4b5b-9831-4c41b85d2f65 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:15.818857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee2a58-4ba2-4b5b-9831-4c41b85d2f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:15.835471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-39ee2a58-4ba2-4b5b-9831-4c41b85d2f65 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:30:15.836669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39ee2a58-4ba2-4b5b-9831-4c41b85d2f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:30:15.900714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 591/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:30:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:30:15.922947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c65151-7371-43be-a74f-82c9f92d923c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:16.018459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:16.022174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c65151-7371-43be-a74f-82c9f92d923c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:16.054919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:16.055703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:16.055988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:16.056263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:16.066683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:30:16.074260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:30:16.075390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:30:16.076630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 597/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:30:16.081588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee2a58-4ba2-4b5b-9831-4c41b85d2f65 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:16.083748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:16.182385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee2a58-4ba2-4b5b-9831-4c41b85d2f65 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:16.214422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:16.215087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:16.215370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:16.215645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:16.216935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:30:16.217239 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:16.218608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:16.226376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 592/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:30:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:16.232886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:16.330884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e4e78-6699-4ada-90ce-7a3b287916e2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:16.364772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:16.365646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:16.365930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:16.366232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] PATH_INFO: `/v3/users/ddf1a6c64532436a87861e208986bd4f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:16.367441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] RBAC: Authorizing `identity:get_user(user_id=ddf1a6c64532436a87861e208986bd4f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:16.369042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:16.370682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 598/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:30:16 2022] GET /identity/v3/users/ddf1a6c64532436a87861e208986bd4f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:16.376477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:16.437075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:16.474511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a71786-696d-47a7-b3cc-3a3a9f569afd None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:16.508038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2f751-13eb-4b94-b520-1803e4bd495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:16.509524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2f751-13eb-4b94-b520-1803e4bd495a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:16.510003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2f751-13eb-4b94-b520-1803e4bd495a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:16.510477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2f751-13eb-4b94-b520-1803e4bd495a None admin] PATH_INFO: `/v3/projects/e0f36ad4b70149bcb304dc804d37b88f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:16.512513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2f751-13eb-4b94-b520-1803e4bd495a None admin] RBAC: Authorizing `identity:get_project(project_id=e0f36ad4b70149bcb304dc804d37b88f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:16.515515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2f751-13eb-4b94-b520-1803e4bd495a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:16.517303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 593/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:30:16 2022] GET /identity/v3/projects/e0f36ad4b70149bcb304dc804d37b88f => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:16.525754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:16.556175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:16.628446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac516-3806-42d8-9c66-fb998fd1a70b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:16.662472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd71d65-de37-4aea-8610-7777418b745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:16.663348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd71d65-de37-4aea-8610-7777418b745e None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:16.663640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd71d65-de37-4aea-8610-7777418b745e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:16.663917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd71d65-de37-4aea-8610-7777418b745e None admin] PATH_INFO: `/v3/projects/e0f36ad4b70149bcb304dc804d37b88f/users/ddf1a6c64532436a87861e208986bd4f/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:16.665903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd71d65-de37-4aea-8610-7777418b745e None admin] RBAC: Authorizing `identity:create_grant(project_id=e0f36ad4b70149bcb304dc804d37b88f, user_id=ddf1a6c64532436a87861e208986bd4f, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:16.667330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd71d65-de37-4aea-8610-7777418b745e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:16.798298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 599/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:30:16 2022] PUT /identity/v3/projects/e0f36ad4b70149bcb304dc804d37b88f/users/ddf1a6c64532436a87861e208986bd4f/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:30:18.790672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99621b4a-b227-44e2-88a6-b4ca1a8c3f48 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:18.791219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99621b4a-b227-44e2-88a6-b4ca1a8c3f48 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:18.791219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99621b4a-b227-44e2-88a6-b4ca1a8c3f48 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:18.792591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 594/1193] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:30:18.798835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fda792-dc87-4276-aea3-0dd48a3bb5b2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:18.799282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fda792-dc87-4276-aea3-0dd48a3bb5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:18.799376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fda792-dc87-4276-aea3-0dd48a3bb5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:18.822512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a2fda792-dc87-4276-aea3-0dd48a3bb5b2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:30:18.823734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a2fda792-dc87-4276-aea3-0dd48a3bb5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:30:18.890601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 600/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:30:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:30:18.914291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99621b4a-b227-44e2-88a6-b4ca1a8c3f48 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:19.012643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99621b4a-b227-44e2-88a6-b4ca1a8c3f48 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:19.042438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:19.047883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:19.048697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:19.048984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:19.049333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:19.059315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:30:19.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:30:19.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:30:19.069494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 595/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:30:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:30:19.074865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fda792-dc87-4276-aea3-0dd48a3bb5b2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:19.093354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:19.172797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fda792-dc87-4276-aea3-0dd48a3bb5b2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:19.205960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:19.206616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:19.206900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:19.207174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:19.207912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:30:19.208215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:19.209607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:19.217407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 601/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:30:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:19.223175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:19.320810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9af157-ed3f-4d9e-b468-16fb5c2358bc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:19.353468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:19.354331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:19.354611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:19.354881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] PATH_INFO: `/v3/users/ddf1a6c64532436a87861e208986bd4f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:19.356052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] RBAC: Authorizing `identity:get_user(user_id=ddf1a6c64532436a87861e208986bd4f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:19.357568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:19.359321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 596/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Jul 15 17:30:19 2022] GET /identity/v3/users/ddf1a6c64532436a87861e208986bd4f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:19.365244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:19.445388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:19.464802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222cc85-6645-48e2-990a-96c629447705 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:19.499643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:19.500579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:19.501001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:19.501284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:19.566890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:19.568342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:19.644660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:19.663661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 602/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:30:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:30:19.670633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:19.727811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:19.766976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91351348-0d99-4740-8af6-3d840c40aefc None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:19.800143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5284f39-aa30-4d23-a50d-224c54e6ba2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:19.801017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5284f39-aa30-4d23-a50d-224c54e6ba2d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:19.801298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5284f39-aa30-4d23-a50d-224c54e6ba2d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:19.801569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5284f39-aa30-4d23-a50d-224c54e6ba2d None admin] PATH_INFO: `/v3/projects/e0f36ad4b70149bcb304dc804d37b88f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:19.802721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5284f39-aa30-4d23-a50d-224c54e6ba2d None admin] RBAC: Authorizing `identity:get_project(project_id=e0f36ad4b70149bcb304dc804d37b88f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:19.804160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5284f39-aa30-4d23-a50d-224c54e6ba2d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:19.805873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 597/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Jul 15 17:30:19 2022] GET /identity/v3/projects/e0f36ad4b70149bcb304dc804d37b88f => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:19.812634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:30:19.910380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622757e-df2d-43c8-ae66-2fcf94e10c2c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:30:19.943231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bd043-938d-4874-8c65-2951f5d4a666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:30:19.943873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bd043-938d-4874-8c65-2951f5d4a666 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:30:19.944161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bd043-938d-4874-8c65-2951f5d4a666 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:30:19.944438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bd043-938d-4874-8c65-2951f5d4a666 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:30:19.945294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bd043-938d-4874-8c65-2951f5d4a666 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=e0f36ad4b70149bcb304dc804d37b88f, user.id=ddf1a6c64532436a87861e208986bd4f) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:30:19.945608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bd043-938d-4874-8c65-2951f5d4a666 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:30:19.947032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bd043-938d-4874-8c65-2951f5d4a666 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:30:19.952819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 603/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:30:19 2022] GET /identity/v3/role_assignments?user.id=ddf1a6c64532436a87861e208986bd4f&scope.project.id=e0f36ad4b70149bcb304dc804d37b88f&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:30:29.796416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=36 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:30:29.951479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=78 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:34.872179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cb3d0-787d-41cb-9d11-4dacceaa740e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:34.872819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cb3d0-787d-41cb-9d11-4dacceaa740e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:34.872819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cb3d0-787d-41cb-9d11-4dacceaa740e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:34.874729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 598/1201] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:31:34.879779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad9564-6d97-4e8f-864e-4843ef601fcd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:34.885338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad9564-6d97-4e8f-864e-4843ef601fcd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:34.885338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad9564-6d97-4e8f-864e-4843ef601fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:34.898850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-45ad9564-6d97-4e8f-864e-4843ef601fcd None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:31:34.900038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-45ad9564-6d97-4e8f-864e-4843ef601fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:31:34.966490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 604/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:31:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 17:31:34.990450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cb3d0-787d-41cb-9d11-4dacceaa740e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:35.112367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cb3d0-787d-41cb-9d11-4dacceaa740e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:35.151805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:35.152472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:35.152789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:35.153059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:35.163504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:35.165007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:35.182902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:35.184139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:35.193117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:35.193117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:35.193117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:35.193117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:35.193117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 599/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:35.193782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad9564-6d97-4e8f-864e-4843ef601fcd None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:35.296754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad9564-6d97-4e8f-864e-4843ef601fcd None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:35.327280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:35.335811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092020f7-2d95-4328-83af-680f852c1d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:35.336482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092020f7-2d95-4328-83af-680f852c1d1e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:35.336799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092020f7-2d95-4328-83af-680f852c1d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:35.337083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092020f7-2d95-4328-83af-680f852c1d1e None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:35.337815 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092020f7-2d95-4328-83af-680f852c1d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:35.338130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092020f7-2d95-4328-83af-680f852c1d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:35.339507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092020f7-2d95-4328-83af-680f852c1d1e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:35.351408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 605/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:31:35.357303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:35.416472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:35.462522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313c641-197c-41d5-a676-91430f80a57d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:35.498074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8d2ed-5b3e-48fa-8ed3-e1837c4bf11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:35.498746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8d2ed-5b3e-48fa-8ed3-e1837c4bf11f None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:35.499051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8d2ed-5b3e-48fa-8ed3-e1837c4bf11f None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:35.499316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8d2ed-5b3e-48fa-8ed3-e1837c4bf11f None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:35.500033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8d2ed-5b3e-48fa-8ed3-e1837c4bf11f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:35.501511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8d2ed-5b3e-48fa-8ed3-e1837c4bf11f None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:35.611206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 600/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Jul 15 17:31:35 2022] POST /identity/v3/users => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:31:37.661137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb3c04-4041-4227-98a0-f6f292f1ab53 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:37.661666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb3c04-4041-4227-98a0-f6f292f1ab53 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:37.661787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb3c04-4041-4227-98a0-f6f292f1ab53 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:37.663093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 606/1206] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:31:37.668442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451428cf-698b-450e-9029-0f7dab7a3a11 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:37.674830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451428cf-698b-450e-9029-0f7dab7a3a11 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:37.674830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451428cf-698b-450e-9029-0f7dab7a3a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:37.686946 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-451428cf-698b-450e-9029-0f7dab7a3a11 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:31:37.688116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-451428cf-698b-450e-9029-0f7dab7a3a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:31:37.757273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 601/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:31:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:31:37.781569 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb3c04-4041-4227-98a0-f6f292f1ab53 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:37.883822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb3c04-4041-4227-98a0-f6f292f1ab53 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:37.919495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:37.920856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:37.928408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:37.928408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:37.931736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:31:37.939437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:37.940665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:37.941794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 607/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:31:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:37.946647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451428cf-698b-450e-9029-0f7dab7a3a11 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:37.985365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:38.042764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451428cf-698b-450e-9029-0f7dab7a3a11 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:38.075752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:38.076435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:38.076752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:38.077033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:38.077774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:38.078082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:38.079460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:38.087211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 602/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:31:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:38.093593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:38.144838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:38.193280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d17d2-b29d-4a03-b3d6-48390f071713 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:38.226909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:38.227732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:38.228019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:38.228298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:38.238612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:38.240041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:38.257164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:38.258237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:38.259339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:38.259339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:38.259339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:38.259339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:38.259339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 608/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:31:38.264917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:38.360498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:38.363242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1141a1-7bb5-42b9-a8b1-baa4d7d47039 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:38.396695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:38.397394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:38.397978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:38.398229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:38.398938 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:38.399241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:38.400658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:38.412030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 603/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:38.417733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:38.517890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f60066-4885-490a-a7da-bb98d00a4c64 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:38.519314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:38.551765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:38.552562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:38.552876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:38.553155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:38.560293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:38.563255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:31:38.577643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:38.578612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:38.579640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 609/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:38.584677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:38.682653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1cec4-e6e3-4473-b3f3-daf1ea4b8424 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:38.716321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:38.717019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:38.717309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:38.717592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:38.718431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:38.718746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:38.720198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:38.737012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 604/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 15 17:31:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:38.742826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:38.841308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30462c92-ec5f-43c6-8086-844e96789c55 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:38.875651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:38.876298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:38.876588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:38.876907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:38.906509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:38.943755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:38.945252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:39.040593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 610/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:31:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:31:39.047037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:39.066007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:39.145387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f2b3e-8cec-4e98-88a8-b791deeb9ed8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:39.179210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:39.179994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:39.180296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:39.180639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:39.190946 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:39.192413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:39.209330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:39.210425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:39.216042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:39.216042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:39.216042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:39.216042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:39.216042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 605/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:39.216388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:39.277842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:39.315586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3127d-ed8d-4eb7-b8e6-04f3ab26613b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:39.351626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:39.352283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:39.352569 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:39.355485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:39.355485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:39.355485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:39.355485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:39.367345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 611/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:39.373007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:39.442630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:39.476769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbf954-a3c1-4be3-b25a-68d4014f9ea8 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:39.511209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:39.512005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:39.512293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:39.512573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:39.519147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:39.520609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:31:39.534914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:39.535995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:39.541700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 606/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:39.543547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:39.644982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba50abd-56e3-466d-ab20-9b1a2523d855 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:39.666621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:39.678086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcb87e-a8a1-43d2-a85a-8c0488c88ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:39.678739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcb87e-a8a1-43d2-a85a-8c0488c88ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:39.679024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcb87e-a8a1-43d2-a85a-8c0488c88ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:39.679300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcb87e-a8a1-43d2-a85a-8c0488c88ca7 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:39.680086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcb87e-a8a1-43d2-a85a-8c0488c88ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:39.680392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcb87e-a8a1-43d2-a85a-8c0488c88ca7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:39.681856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcb87e-a8a1-43d2-a85a-8c0488c88ca7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:39.694078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 612/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:31:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:39.700477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:39.802113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db5808-e977-4c62-924d-ad1e90691be5 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:39.830243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:39.835779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f59325-ed7b-4e47-b6b3-f06339489d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:39.836429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59325-ed7b-4e47-b6b3-f06339489d5d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:39.836748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59325-ed7b-4e47-b6b3-f06339489d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:39.837028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59325-ed7b-4e47-b6b3-f06339489d5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:39.837911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f59325-ed7b-4e47-b6b3-f06339489d5d None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=b79bed7026ac4124a1228bc06f298c34) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:39.838226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f59325-ed7b-4e47-b6b3-f06339489d5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:39.839670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f59325-ed7b-4e47-b6b3-f06339489d5d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:39.845702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 607/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:31:39 2022] GET /identity/v3/role_assignments?user.id=b79bed7026ac4124a1228bc06f298c34&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:41.874363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8ea38-7269-4cb3-afa8-e036a0830e86 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:41.874897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8ea38-7269-4cb3-afa8-e036a0830e86 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:41.874897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8ea38-7269-4cb3-afa8-e036a0830e86 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:41.876414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 613/1220] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:31:41.883632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c778-4580-4ca9-b36a-64ff011a096a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:41.884110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c778-4580-4ca9-b36a-64ff011a096a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:41.884399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c778-4580-4ca9-b36a-64ff011a096a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:41.907801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7325c778-4580-4ca9-b36a-64ff011a096a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:31:41.909079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7325c778-4580-4ca9-b36a-64ff011a096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:31:41.975143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 608/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:31:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:31:41.997336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8ea38-7269-4cb3-afa8-e036a0830e86 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:42.098285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8ea38-7269-4cb3-afa8-e036a0830e86 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:42.132237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:42.132982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:42.133272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:42.133547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:42.144042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:31:42.151988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:42.153297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:31:42.154155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:42.156968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 614/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:31:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:42.159406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c778-4580-4ca9-b36a-64ff011a096a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:42.257171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c778-4580-4ca9-b36a-64ff011a096a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:42.281156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:42.291692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:42.292369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:42.292673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:42.292949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:42.293694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] RBAC: Adding query filter params (name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:42.294001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:42.295373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:42.303922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 609/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:31:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:42.309699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:42.353541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:42.410731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b581-c887-4940-a7c3-95190aa25f18 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:42.443986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:42.444797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:42.445089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:42.445370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:42.456428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:42.457906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:42.475385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:42.476498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:42.481841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:42.481841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:42.481841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:42.481841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:42.481841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 615/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:31:42.482620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:42.581935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51805a4e-47a8-4995-be13-7a5c3c136eca None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:42.614728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:42.615381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:42.615666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:42.615942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:42.616688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:42.616997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:42.618517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:42.630430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 610/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:42.636158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:42.728960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:42.735457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe432-3d11-4481-882c-08742fe86826 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:42.769295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:42.770090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:42.770373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:42.770646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:42.778159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:42.779573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:31:42.794102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:42.795026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:42.795987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 616/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:42.801299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:42.848223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:42.898991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574a0dc-2382-498e-afe6-3ff46d85567c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:42.931395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:42.932062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:42.932352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:42.932657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:42.933493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:42.933811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:42.935260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:42.952914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 611/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 15 17:31:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:42.959239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:43.058622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1de62d-6ba5-4ddd-8713-9739ebf90182 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:43.092573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:43.093379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:43.093665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:43.093941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:43.104730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:43.106172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:43.123361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:43.129243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:43.130351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:43.130351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:43.130351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:43.130351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:43.130351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 617/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:43.130935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:43.228670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a41d2-f57e-4c90-827e-ee2fcc3c1170 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:43.231069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:43.263311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:43.263974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:43.264266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:43.264535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:43.265296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:43.265601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:43.266981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:43.278798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 612/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:31:43.284519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:43.293874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:43.382873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc033e-90e0-435a-8a5c-9e53db2ff96f None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:43.415574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:43.416364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:43.416676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:43.416952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:43.424000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:43.425461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:31:43.440396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:43.441472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:43.442478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 618/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:31:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:43.448790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:43.551280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff3cd-846c-4215-8383-04c6558cdb57 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:43.583992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9369d-73b7-4e9d-93b8-3a29e5443dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:43.584671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9369d-73b7-4e9d-93b8-3a29e5443dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:43.584961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9369d-73b7-4e9d-93b8-3a29e5443dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:43.585242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9369d-73b7-4e9d-93b8-3a29e5443dd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:43.586036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9369d-73b7-4e9d-93b8-3a29e5443dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:43.586348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9369d-73b7-4e9d-93b8-3a29e5443dd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:43.587803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9369d-73b7-4e9d-93b8-3a29e5443dd8 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:43.599562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 613/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:31:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:43.605772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:43.671963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:43.705382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800dcdd-5934-49e9-bb85-501f5aa557af None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:43.738651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074ccd5-a896-459e-8be0-450784294fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:43.739568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ccd5-a896-459e-8be0-450784294fd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:43.739849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ccd5-a896-459e-8be0-450784294fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:43.740123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ccd5-a896-459e-8be0-450784294fd7 None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/b79bed7026ac4124a1228bc06f298c34/roles/22cdd113399a479bb54fcb86fc3f2a15` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:43.754250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074ccd5-a896-459e-8be0-450784294fd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=b79bed7026ac4124a1228bc06f298c34, role_id=22cdd113399a479bb54fcb86fc3f2a15)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:43.755683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074ccd5-a896-459e-8be0-450784294fd7 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:43.901899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 619/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:31:43 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/b79bed7026ac4124a1228bc06f298c34/roles/22cdd113399a479bb54fcb86fc3f2a15 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:31:45.900055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f85361-74fb-4088-bbe6-ac7148330294 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:45.900628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f85361-74fb-4088-bbe6-ac7148330294 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:45.900746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f85361-74fb-4088-bbe6-ac7148330294 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:45.902093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 614/1233] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:31:45.908037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bf125-f756-4a0d-81e8-99bd101c3d29 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:45.908437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bf125-f756-4a0d-81e8-99bd101c3d29 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:45.908586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bf125-f756-4a0d-81e8-99bd101c3d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:45.925352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e72bf125-f756-4a0d-81e8-99bd101c3d29 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:31:45.926528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e72bf125-f756-4a0d-81e8-99bd101c3d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:31:45.992801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 620/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:31:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:31:46.015944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f85361-74fb-4088-bbe6-ac7148330294 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:46.041046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:46.115548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f85361-74fb-4088-bbe6-ac7148330294 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:46.148561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:46.149378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:46.149661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:46.149937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:46.160736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:31:46.168272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:46.169437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:46.170298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 615/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Jul 15 17:31:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:46.176410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bf125-f756-4a0d-81e8-99bd101c3d29 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:46.222843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:46.277229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bf125-f756-4a0d-81e8-99bd101c3d29 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:46.310037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:46.310696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:46.310978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:46.311252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:46.312041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] RBAC: Adding query filter params (name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:46.312344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:46.313729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:46.322144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 621/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Jul 15 17:31:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:46.327822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:46.411882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:46.427255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c7cd3-8d39-45e8-b2b5-0972b01ec375 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:46.460934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:46.461726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:46.462014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:46.462294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:46.472824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:46.474289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:46.491467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:46.492647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:46.493966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:46.493966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:46.493966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:46.493966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:46.493966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 616/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:46.498833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:46.598432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f128b4-4cbd-4010-aaf7-45c163fb312e None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:46.607537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:46.632018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:46.632709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:46.632998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:46.633277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:46.633998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:46.634306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:46.635669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:46.647923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 622/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:31:46.653545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:46.751057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f526e7f-676c-4bda-a297-c2d3607b8431 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:46.785196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:46.785984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:46.786272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:46.786550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:46.789178 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:46.793609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:46.795059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:31:46.810235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:46.811422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:46.813408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 617/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:46.819031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:46.926454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611f58-21af-4157-b856-9c537b2f4772 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:46.960042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:46.960741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:46.961019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:46.961292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:46.962121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:46.962427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:46.963847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:46.981780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 623/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 15 17:31:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:46.987581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:47.087547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162443a-4abc-45e8-9d03-eb76afc0810a None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:47.130517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:47.131162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:47.131444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:47.131717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:47.192403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:47.198756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:47.200231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:47.294646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 618/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:31:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:31:47.301108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:47.330462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:47.403137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d631672-e100-4723-ad85-78666e43beb6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:47.438442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:47.439231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:47.439511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:47.439785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:47.449814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:47.451250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:47.468930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:47.470011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:47.471627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:47.471627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:47.471627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:47.471627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:47.471627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 624/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:47.476381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:47.569857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:47.575804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56448f-9470-43ff-ace7-9b9b2ecfa056 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:47.608573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:47.609265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:47.609556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:47.609837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:47.610561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:47.610872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:47.612902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:47.624145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 619/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:47.629895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:47.708866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:47.729797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f004fb-03b5-446f-8708-b0eef0e168f9 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:47.763270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:47.764052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:47.764334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:47.764627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:47.771560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:47.773277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:31:47.787446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:47.792118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:47.793363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 625/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:31:47.794688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:47.892009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16978a1f-a641-4e4a-b6e9-3ba4b53bccde None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:47.925928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293904bf-293a-40dc-89e3-7519718eff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:47.926582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293904bf-293a-40dc-89e3-7519718eff19 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:47.926872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293904bf-293a-40dc-89e3-7519718eff19 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:47.927152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293904bf-293a-40dc-89e3-7519718eff19 None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:47.927935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293904bf-293a-40dc-89e3-7519718eff19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:47.928251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293904bf-293a-40dc-89e3-7519718eff19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:47.929723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293904bf-293a-40dc-89e3-7519718eff19 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:47.941634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 620/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:47.947750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:48.048580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9da1bf-4bb7-4a1e-9389-632074ed9722 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:48.086193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261248fd-db62-4481-88c8-854ca49edfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:48.086849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261248fd-db62-4481-88c8-854ca49edfaa None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:48.087135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261248fd-db62-4481-88c8-854ca49edfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:48.087414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261248fd-db62-4481-88c8-854ca49edfaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:48.088246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261248fd-db62-4481-88c8-854ca49edfaa None admin] RBAC: Adding query filter params (role.id=22cdd113399a479bb54fcb86fc3f2a15, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=b79bed7026ac4124a1228bc06f298c34) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:48.088565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261248fd-db62-4481-88c8-854ca49edfaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:48.090096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261248fd-db62-4481-88c8-854ca49edfaa None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:48.096528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 626/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:31:47 2022] GET /identity/v3/role_assignments?user.id=b79bed7026ac4124a1228bc06f298c34&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=22cdd113399a479bb54fcb86fc3f2a15 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:50.171983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5997da-832b-4806-a406-3116f68976f1 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:50.172568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5997da-832b-4806-a406-3116f68976f1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:50.172568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5997da-832b-4806-a406-3116f68976f1 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:50.174298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 621/1247] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:31:50.179596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396cac-a18a-47f7-8789-6ea65f668e11 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:50.180004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396cac-a18a-47f7-8789-6ea65f668e11 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:50.181999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396cac-a18a-47f7-8789-6ea65f668e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:50.185211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:50.197811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f7396cac-a18a-47f7-8789-6ea65f668e11 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:31:50.199078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f7396cac-a18a-47f7-8789-6ea65f668e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:31:50.264455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 627/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:31:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:31:50.288954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5997da-832b-4806-a406-3116f68976f1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:50.306680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:50.388973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5997da-832b-4806-a406-3116f68976f1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:50.426110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:50.426840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:50.427126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:50.427398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:50.438860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:31:50.446943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:50.448068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:50.448959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 622/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:31:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:50.454364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396cac-a18a-47f7-8789-6ea65f668e11 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:50.553056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7396cac-a18a-47f7-8789-6ea65f668e11 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:50.582648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:50.587501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:50.588167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:50.588453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:50.588761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:50.589503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:50.589809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:50.591165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:50.599092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 628/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:50.604936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:50.680359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:50.702356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495e5c-ac3d-46b7-b228-2faa7d5d59f3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:50.734391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:50.735172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:50.735458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:50.735738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:50.746127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:50.747579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:50.765789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:50.766923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:50.768874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:50.768874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:50.768874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:50.768874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:50.768874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 623/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:50.773611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:50.872117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e723da-8317-4351-903c-d5c42644c12b None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:50.906003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:50.906663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:50.906947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:50.907223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:50.907972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:50.908280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:50.909685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:50.921275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 629/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:50.927470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:51.025366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8767dac-8933-4fa4-ba9e-d5e8944b477d None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:51.051040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:51.058582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:51.059375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:51.059657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:51.059980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:51.068652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:51.070108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:31:51.084170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:51.085337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:51.086673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 624/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:51.091287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:51.149199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:51.191924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa5c87-86dc-4b82-90d5-217791016353 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:51.233268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:51.233938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:51.234224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:51.234499 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:51.235339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:51.235650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:51.237725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:51.255661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 630/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 15 17:31:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:51.261660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:51.361963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ca7f-5079-4726-a5b8-0359ee1b412e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:51.395411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:51.396058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:51.396349 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:51.396656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:51.461624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:51.473973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:51.475470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:51.571753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 625/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:31:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 17:31:51.577678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:51.675327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21ad23-eec7-46d7-908d-56e7eebce1cb None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:51.687855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:51.708523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:51.709337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:51.709668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:51.709954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:51.719776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:51.721801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:51.738799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:51.739959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:51.741106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:51.741106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:51.741106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:51.741106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:51.741106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 631/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:51.745747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:51.831928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:51.843257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbc942-470b-4068-ba9d-0b18e12a9ce3 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:51.875924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:51.877116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:51.877407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:51.877685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:51.878405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:51.878715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:51.880104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:51.891526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 626/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:51.898132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:51.995665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cef0f-29e5-41d6-a2d4-1b62014f8883 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:52.029821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:52.030672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:52.030939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:52.031218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:52.037803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:52.039242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:31:52.052786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:52.053854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:52.054954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 632/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:52.059792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:52.160335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b11e-1235-4464-99b4-fad38d2f0204 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:52.193510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adae68-9b0a-445d-bbe6-894194de160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:52.194157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adae68-9b0a-445d-bbe6-894194de160c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:52.194445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adae68-9b0a-445d-bbe6-894194de160c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:52.194725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adae68-9b0a-445d-bbe6-894194de160c None admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:52.195508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adae68-9b0a-445d-bbe6-894194de160c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:52.195822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adae68-9b0a-445d-bbe6-894194de160c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:52.197297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adae68-9b0a-445d-bbe6-894194de160c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:52.208922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 627/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:52.215014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:52.243156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:52.316096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b3de2-6c68-40be-88df-a7408c0a29ee None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:52.350801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f58f2-5858-4d4c-84d8-551cb2e889db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:52.351497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f58f2-5858-4d4c-84d8-551cb2e889db None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:52.351784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f58f2-5858-4d4c-84d8-551cb2e889db None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:52.352065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f58f2-5858-4d4c-84d8-551cb2e889db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:52.352931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f58f2-5858-4d4c-84d8-551cb2e889db None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=b79bed7026ac4124a1228bc06f298c34) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:52.353247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f58f2-5858-4d4c-84d8-551cb2e889db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:52.354678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f58f2-5858-4d4c-84d8-551cb2e889db None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:52.360014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 633/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:31:52 2022] GET /identity/v3/role_assignments?user.id=b79bed7026ac4124a1228bc06f298c34&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:54.337615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573940b-1268-4cb9-bbf9-4b9435a83361 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:54.338135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573940b-1268-4cb9-bbf9-4b9435a83361 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:54.338249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573940b-1268-4cb9-bbf9-4b9435a83361 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:54.339550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 628/1261] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:31:54.345037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c5d53-b6d6-4739-aab5-f43ef1f8050a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:54.345979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c5d53-b6d6-4739-aab5-f43ef1f8050a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:54.346166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c5d53-b6d6-4739-aab5-f43ef1f8050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:54.367910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c59c5d53-b6d6-4739-aab5-f43ef1f8050a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:31:54.369121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c59c5d53-b6d6-4739-aab5-f43ef1f8050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:31:54.433811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 634/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:31:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:31:54.455764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573940b-1268-4cb9-bbf9-4b9435a83361 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:54.466677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:54.555869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573940b-1268-4cb9-bbf9-4b9435a83361 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:54.590197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:54.590976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:54.591264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:54.591553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:54.602199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:31:54.609719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:54.613763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:54.614837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 629/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:54.617180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c5d53-b6d6-4739-aab5-f43ef1f8050a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:54.635067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:54.713918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c5d53-b6d6-4739-aab5-f43ef1f8050a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:54.747762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:54.748456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:54.748767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:54.749043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:54.749781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:54.750085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:54.752165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:54.759944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 635/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:54.765798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:54.830116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:54.861513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b6f7a-b51a-4359-9bce-ce4505216e83 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:54.893494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:54.894270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:54.894550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:54.894821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:54.905711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:54.907187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:54.924926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:54.925972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:54.927283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:54.927283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:54.927283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:54.927283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:54.927283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 630/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:54.932007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:55.012615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:55.031260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7f359-f596-440d-9797-bb9f70e28650 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:55.064260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:55.064945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:55.065233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:55.065511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:55.066228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:55.066537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:55.067929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:55.079191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 636/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:55.084979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:55.184740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aac1ce-bdd0-462d-a323-02cf50432e7c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:55.208409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:55.222746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:55.223547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:55.223831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:55.224107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:55.232385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:55.233866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:31:55.249053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:55.250205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:55.251701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 631/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:55.256242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:55.356461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee2f30-001b-4672-adf7-a1c98348bea3 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:55.389831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:55.390501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:55.390785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:55.391061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:55.391894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:55.392756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:55.394181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:55.403013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:55.413814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 637/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 15 17:31:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:55.418282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:55.516820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46417b41-6bdc-4c87-96de-39eb2e20d68e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:55.550856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:55.551673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:55.551958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:55.552233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:55.562609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:55.564052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:55.581191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:55.582254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:55.583333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:55.583333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:55.583333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:55.583333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:55.583333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 632/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 17:31:55.589193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:55.688122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f19141-4b67-4ca1-bf33-ff1193b0d8de None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:55.721790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:55.722441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:55.722724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:55.723063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:55.723756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:55.724059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:55.725434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:55.736588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 638/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:55.742408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:55.778240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:55.840939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79432b0e-2c5f-4485-b918-1373a034dd8b None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:55.873043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:55.873824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:55.874119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:55.874395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:55.881448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:55.882889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:31:55.896884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:55.898187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:55.899274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 633/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:31:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:55.904213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:55.962254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:56.002375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5342741-5630-4093-90b5-2448e830c30c None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:56.035174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac86c0-ee46-4e58-94a4-80c53b6a94ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:56.035832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac86c0-ee46-4e58-94a4-80c53b6a94ad None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:56.036119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac86c0-ee46-4e58-94a4-80c53b6a94ad None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:56.036396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac86c0-ee46-4e58-94a4-80c53b6a94ad None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:56.037229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac86c0-ee46-4e58-94a4-80c53b6a94ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:56.037532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac86c0-ee46-4e58-94a4-80c53b6a94ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:56.038954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac86c0-ee46-4e58-94a4-80c53b6a94ad None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:56.052128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 639/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:31:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:56.058147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:56.149875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:56.158408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3953d-d4a4-487f-a680-0a702a11d2ec None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:56.192117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b63b6-9888-4e40-8d0f-121c06608a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:56.193033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b63b6-9888-4e40-8d0f-121c06608a5e None admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:56.193321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b63b6-9888-4e40-8d0f-121c06608a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:56.193601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b63b6-9888-4e40-8d0f-121c06608a5e None admin] PATH_INFO: `/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/b79bed7026ac4124a1228bc06f298c34/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:56.195579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b63b6-9888-4e40-8d0f-121c06608a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=d294fe03b8be4a2f962e02624c929ca7, user_id=b79bed7026ac4124a1228bc06f298c34, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:56.197057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b63b6-9888-4e40-8d0f-121c06608a5e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:56.279447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 634/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Jul 15 17:31:56 2022] PUT /identity/v3/projects/d294fe03b8be4a2f962e02624c929ca7/users/b79bed7026ac4124a1228bc06f298c34/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:31:58.272556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f77ef-1d81-48eb-82f7-81751d78a086 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:58.273049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f77ef-1d81-48eb-82f7-81751d78a086 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:58.273164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f77ef-1d81-48eb-82f7-81751d78a086 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:58.274504 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 640/1274] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:31:58.279732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee236b8-7baf-4f48-ae3d-4a2828cc2ad0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:58.280133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee236b8-7baf-4f48-ae3d-4a2828cc2ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:58.280317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee236b8-7baf-4f48-ae3d-4a2828cc2ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:58.296801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1ee236b8-7baf-4f48-ae3d-4a2828cc2ad0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:31:58.297988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1ee236b8-7baf-4f48-ae3d-4a2828cc2ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:31:58.364336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 635/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:31:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:31:58.388000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f77ef-1d81-48eb-82f7-81751d78a086 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:58.488271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f77ef-1d81-48eb-82f7-81751d78a086 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:58.506889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:58.523448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:58.524235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:58.524520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:58.524828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:58.534623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 17:31:58.542191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:58.545538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:58.546471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 641/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Jul 15 17:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:58.550003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee236b8-7baf-4f48-ae3d-4a2828cc2ad0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:58.631108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:58.648603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee236b8-7baf-4f48-ae3d-4a2828cc2ad0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:58.682713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:58.683379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:58.683671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:58.683948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:58.684722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:58.685030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:58.686437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:58.694327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 636/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Jul 15 17:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:58.700197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:58.799251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b0747-215e-42f4-97a3-9c3827293bdf None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:58.833017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:58.833795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:58.834080 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:58.834357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:58.844615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:58.846088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:58.863072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:58.864158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:58.865302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:58.865302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:58.865302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:58.865302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:58.865302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 642/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:58.869982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:58.969084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee74b3-8238-4071-bc68-c2ebad8feb77 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:59.000973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:59.006094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:59.006094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:59.006094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:59.006094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:59.006094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:59.006094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:59.019024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 637/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:59.024976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:59.055777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:59.125323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fe82b-261a-4eb2-bc55-586173df1a96 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:59.159416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:59.160220 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:59.160512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:59.160831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:59.168166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:59.169638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 17:31:59.184331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:59.185497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:31:59.186650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 643/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:59.192151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:59.195947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:59.288153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144fd5-84a4-4367-9d73-f90e872bea55 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:59.322164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:59.322164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:59.322164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:59.322491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:59.323291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:59.323603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:59.325108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:59.341483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 638/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Jul 15 17:31:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:59.347241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:59.438086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:59.447652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3b795-a551-4f59-a84e-5c8f37c018a6 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:59.480073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:59.480753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:59.481040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:59.481318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:59.547565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:59.549797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:59.636499 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:59.646927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 644/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Jul 15 17:31:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 17:31:59.653059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:59.713537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:31:59.750000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d1154-9a0d-4124-ab16-29a353a391e9 None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:59.783668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:59.784438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:59.784760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:59.785043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:59.795356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:59.796842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 17:31:59.814747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:31:59.815780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:31:59.817001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 17:31:59.817001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 17:31:59.817001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 17:31:59.817001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:31:59.817001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 639/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:31:59.822004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:31:59.920678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1ec0d-615b-4df3-8648-e1f274e3c287 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:31:59.955340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:31:59.956458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:31:59.957005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:31:59.957487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:31:59.958848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:31:59.959294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:31:59.960707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:31:59.972142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 645/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Jul 15 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:31:59.978036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:00.077146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3966-f18a-4896-acf2-03ca64c1e2ee None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:00.091699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:00.111921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:00.120075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:00.120075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:00.120075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:00.120295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:00.121773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 17:32:00.135766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application with Lock( Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:32:00.137486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: ERROR keystone.server.flask.application Jul 15 17:32:00.139311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 640/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Jul 15 17:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:32:00.144505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:00.194843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:00.244477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0330ca-dcfd-49a2-8385-3c5f998daa23 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:00.281686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d9a9f-26c9-426f-9e62-ed1bb9efcc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:00.282346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9a9f-26c9-426f-9e62-ed1bb9efcc90 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:00.283178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9a9f-26c9-426f-9e62-ed1bb9efcc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:00.283434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9a9f-26c9-426f-9e62-ed1bb9efcc90 None admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:00.284219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d9a9f-26c9-426f-9e62-ed1bb9efcc90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:32:00.284520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d9a9f-26c9-426f-9e62-ed1bb9efcc90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:00.285964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d9a9f-26c9-426f-9e62-ed1bb9efcc90 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:00.297846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 646/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Jul 15 17:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:32:00.304542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:00.399389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee9bdf-fc66-461e-913f-ae611b0ae54c None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:00.433558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec03d8-0935-4132-9bb6-1a705c26d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:00.434209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec03d8-0935-4132-9bb6-1a705c26d302 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:00.434496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec03d8-0935-4132-9bb6-1a705c26d302 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:00.434776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec03d8-0935-4132-9bb6-1a705c26d302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:00.435675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec03d8-0935-4132-9bb6-1a705c26d302 None admin] RBAC: Adding query filter params (role.id=2deb39ad6d084e90b2f50fbab45f1df4, scope.project.id=d294fe03b8be4a2f962e02624c929ca7, user.id=b79bed7026ac4124a1228bc06f298c34) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:32:00.435992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec03d8-0935-4132-9bb6-1a705c26d302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:00.437466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec03d8-0935-4132-9bb6-1a705c26d302 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:00.443190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 641/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Jul 15 17:32:00 2022] GET /identity/v3/role_assignments?user.id=b79bed7026ac4124a1228bc06f298c34&scope.project.id=d294fe03b8be4a2f962e02624c929ca7&role.id=2deb39ad6d084e90b2f50fbab45f1df4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:32:02.465406 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9944-fb26-40a5-ac9b-27fcb68da692 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:02.465961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9944-fb26-40a5-ac9b-27fcb68da692 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:02.465961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9944-fb26-40a5-ac9b-27fcb68da692 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:02.467397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 647/1288] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:32:02.472929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e54e-b147-4165-9d1d-442b6994d195 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:02.473331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e54e-b147-4165-9d1d-442b6994d195 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:02.473482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e54e-b147-4165-9d1d-442b6994d195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:02.490818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-28c0e54e-b147-4165-9d1d-442b6994d195 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:32:02.495323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-28c0e54e-b147-4165-9d1d-442b6994d195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:32:02.520518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:02.556955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 642/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:32:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:32:02.578818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d9944-fb26-40a5-ac9b-27fcb68da692 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:02.680333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d9944-fb26-40a5-ac9b-27fcb68da692 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:02.691112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:02.714293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:02.715087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:02.715375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:02.715654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:02.716305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:02.717713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:32:02.730156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:32:02.731397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:02.732580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 648/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:32:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:32:02.737761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0e54e-b147-4165-9d1d-442b6994d195 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:02.833549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0e54e-b147-4165-9d1d-442b6994d195 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:02.866383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc402a4-a0ae-4bb4-afea-9e2b14cc97b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:02.867037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc402a4-a0ae-4bb4-afea-9e2b14cc97b6 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:02.867322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc402a4-a0ae-4bb4-afea-9e2b14cc97b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:02.867598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc402a4-a0ae-4bb4-afea-9e2b14cc97b6 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:02.868299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc402a4-a0ae-4bb4-afea-9e2b14cc97b6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:32:02.868633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc402a4-a0ae-4bb4-afea-9e2b14cc97b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:02.870800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc402a4-a0ae-4bb4-afea-9e2b14cc97b6 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:02.875659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 643/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 15 17:32:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:32:02.887583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:02.981432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb97f7-f602-4e7e-b8cc-1315503b3fe2 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:03.014006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f96de-fe7f-4ff2-bf2e-41a31f0e85af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:03.014658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f96de-fe7f-4ff2-bf2e-41a31f0e85af None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:03.014942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f96de-fe7f-4ff2-bf2e-41a31f0e85af None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:03.015218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f96de-fe7f-4ff2-bf2e-41a31f0e85af None admin] PATH_INFO: `/v3/services` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:03.015919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f96de-fe7f-4ff2-bf2e-41a31f0e85af None admin] RBAC: Adding query filter params (type=placement) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:32:03.016226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f96de-fe7f-4ff2-bf2e-41a31f0e85af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:03.018149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f96de-fe7f-4ff2-bf2e-41a31f0e85af None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:03.022776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 649/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 15 17:32:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:32:05.067942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280983f-1aec-46f7-b583-f1e4271d6a44 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:05.068504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280983f-1aec-46f7-b583-f1e4271d6a44 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:05.068504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280983f-1aec-46f7-b583-f1e4271d6a44 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:05.069915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 644/1293] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:32:05.075156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683fdec-ffea-4ad9-a758-63cbdac3452d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:05.075567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683fdec-ffea-4ad9-a758-63cbdac3452d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:05.075692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683fdec-ffea-4ad9-a758-63cbdac3452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:05.092535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e683fdec-ffea-4ad9-a758-63cbdac3452d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:32:05.093739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e683fdec-ffea-4ad9-a758-63cbdac3452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:32:05.123039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:05.159950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 650/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:32:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:32:05.181695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280983f-1aec-46f7-b583-f1e4271d6a44 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:05.212961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:05.285154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280983f-1aec-46f7-b583-f1e4271d6a44 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:05.317853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33fa22-40d4-4f1f-8027-b6296887370e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:05.318562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33fa22-40d4-4f1f-8027-b6296887370e None admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:05.318792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33fa22-40d4-4f1f-8027-b6296887370e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:05.319069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33fa22-40d4-4f1f-8027-b6296887370e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:05.319798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33fa22-40d4-4f1f-8027-b6296887370e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:05.321206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33fa22-40d4-4f1f-8027-b6296887370e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:05.421482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 645/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Jul 15 17:32:05 2022] POST /identity/v3/services => generated 246 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:32:07.442524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab118-7c31-4ed1-b26d-9c58e628c5e1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:07.443067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab118-7c31-4ed1-b26d-9c58e628c5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:07.443067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab118-7c31-4ed1-b26d-9c58e628c5e1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:07.444461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 651/1296] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:32:07.449717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed710224-71f4-4887-a26c-3be5041ba540 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:07.450122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed710224-71f4-4887-a26c-3be5041ba540 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:07.450282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed710224-71f4-4887-a26c-3be5041ba540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:07.466797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ed710224-71f4-4887-a26c-3be5041ba540 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:32:07.467972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ed710224-71f4-4887-a26c-3be5041ba540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:32:07.540474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 646/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:32:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:32:07.564458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab118-7c31-4ed1-b26d-9c58e628c5e1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:07.666883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab118-7c31-4ed1-b26d-9c58e628c5e1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:07.700903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:07.701687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:07.701968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:07.702240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:07.702879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:07.704250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:32:07.715944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:32:07.724969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:07.726027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 652/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:32:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:32:07.726985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed710224-71f4-4887-a26c-3be5041ba540 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:07.743050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:07.822697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed710224-71f4-4887-a26c-3be5041ba540 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:07.856456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:07.857186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:07.858382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:07.858700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:07.859407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:32:07.859722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:07.861655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:07.866717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 647/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 15 17:32:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:32:07.872876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:07.904134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:07.970763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28c3e2-a618-418d-8359-92aaadbdb983 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:08.003324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c327583-7d98-4730-b314-3c1916ffff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:08.004231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c327583-7d98-4730-b314-3c1916ffff2c None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:08.005113 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c327583-7d98-4730-b314-3c1916ffff2c None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:08.005396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c327583-7d98-4730-b314-3c1916ffff2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:08.006196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c327583-7d98-4730-b314-3c1916ffff2c None admin] RBAC: Adding query filter params (interface=public, service_id=8e66aa256aec4cbc97d6e8ee71a3a2f1, region_id=RegionOne) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:32:08.006507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c327583-7d98-4730-b314-3c1916ffff2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:08.007879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c327583-7d98-4730-b314-3c1916ffff2c None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:08.013175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 653/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Jul 15 17:32:07 2022] GET /identity/v3/endpoints?service_id=8e66aa256aec4cbc97d6e8ee71a3a2f1&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:32:08.019374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:08.099065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:08.116283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d817e4-93fd-4a4f-8c2a-fc7ed5be4b1e None admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:08.149461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e469c3-9d2e-4fa5-9fd4-38dc7a35e932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:08.150127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e469c3-9d2e-4fa5-9fd4-38dc7a35e932 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:08.150413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e469c3-9d2e-4fa5-9fd4-38dc7a35e932 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:08.150694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e469c3-9d2e-4fa5-9fd4-38dc7a35e932 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:08.151670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e469c3-9d2e-4fa5-9fd4-38dc7a35e932 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:08.152788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e469c3-9d2e-4fa5-9fd4-38dc7a35e932 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:08.160007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 648/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Jul 15 17:32:08 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:32:10.191036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13866b4-c06c-4e53-920f-02da1e3e7cd9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:10.191621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13866b4-c06c-4e53-920f-02da1e3e7cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:10.191621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13866b4-c06c-4e53-920f-02da1e3e7cd9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:10.193012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 654/1302] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:32:10.198195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8756-aebd-48cf-a045-7050b6163ac4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:10.198629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8756-aebd-48cf-a045-7050b6163ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:10.198776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8756-aebd-48cf-a045-7050b6163ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:10.215760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e9dd8756-aebd-48cf-a045-7050b6163ac4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:32:10.216973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e9dd8756-aebd-48cf-a045-7050b6163ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:32:10.285467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 649/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:32:10 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 17:32:10.308121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13866b4-c06c-4e53-920f-02da1e3e7cd9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:10.402732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:10.409682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13866b4-c06c-4e53-920f-02da1e3e7cd9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:10.443118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:10.443848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:10.444133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:10.444410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:10.445205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:10.446565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 17:32:10.458895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 17:32:10.469202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application with Lock( Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self._enter() Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application return self.creator() Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application created_value = creator( Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: ERROR keystone.server.flask.application Jul 15 17:32:10.470130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 655/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Jul 15 17:32:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 17:32:10.470991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd8756-aebd-48cf-a045-7050b6163ac4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:10.540866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:10.563201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd8756-aebd-48cf-a045-7050b6163ac4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:10.596344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc48b0a-bc5e-49cc-99e3-8cbec64b7273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:10.597069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc48b0a-bc5e-49cc-99e3-8cbec64b7273 None admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:10.597361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc48b0a-bc5e-49cc-99e3-8cbec64b7273 None admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:10.597643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc48b0a-bc5e-49cc-99e3-8cbec64b7273 None admin] PATH_INFO: `/v3/services` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:10.598352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc48b0a-bc5e-49cc-99e3-8cbec64b7273 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:32:10.598661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc48b0a-bc5e-49cc-99e3-8cbec64b7273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:10.600040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc48b0a-bc5e-49cc-99e3-8cbec64b7273 None admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:10.605129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 650/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Jul 15 17:32:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:32:10.611463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:32:10.711087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77537-aee5-4183-9fcc-a79daed2ef23 None admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:32:10.745141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd151a49-9d0d-4f3e-b594-58ff6cdaaefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:32:10.745820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd151a49-9d0d-4f3e-b594-58ff6cdaaefe None admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:32:10.746108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd151a49-9d0d-4f3e-b594-58ff6cdaaefe None admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:32:10.746386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd151a49-9d0d-4f3e-b594-58ff6cdaaefe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:32:10.747129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd151a49-9d0d-4f3e-b594-58ff6cdaaefe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:32:10.748491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd151a49-9d0d-4f3e-b594-58ff6cdaaefe None admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:32:10.838898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 656/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Jul 15 17:32:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:32:20.655545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=30 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:32:20.834189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:16.390797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcb70a-bb08-4213-809a-301171b24a69 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:16.391383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcb70a-bb08-4213-809a-301171b24a69 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:16.391383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcb70a-bb08-4213-809a-301171b24a69 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:16.393158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 651/1307] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:16.398603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd14f9-c1c3-49cc-bd0c-8cf99868cb69 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:16.399180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd14f9-c1c3-49cc-bd0c-8cf99868cb69 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:16.399294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd14f9-c1c3-49cc-bd0c-8cf99868cb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:16.494624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b7dd14f9-c1c3-49cc-bd0c-8cf99868cb69 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:16.495699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b7dd14f9-c1c3-49cc-bd0c-8cf99868cb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:16.618187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 657/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:16.673693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc994c2d-65fe-48f5-bb51-858789b1679f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:16.674408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc994c2d-65fe-48f5-bb51-858789b1679f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:16.674824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc994c2d-65fe-48f5-bb51-858789b1679f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:16.676731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 652/1309] 10.222.0.57 () {54 vars in 964 bytes} [Fri Jul 15 17:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:16.683146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8a313-84c1-4db9-8d82-84c63fdd8998 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:16.683448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8a313-84c1-4db9-8d82-84c63fdd8998 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:16.683735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8a313-84c1-4db9-8d82-84c63fdd8998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:16.777168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-90f8a313-84c1-4db9-8d82-84c63fdd8998 None None] MFA Rules not processed for user `35d3862fe9e846f18d8bc3a0fa5e993a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:16.778408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-90f8a313-84c1-4db9-8d82-84c63fdd8998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:16.803252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:16.867987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 658/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Fri Jul 15 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:16.879378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc994c2d-65fe-48f5-bb51-858789b1679f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:16.881644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc994c2d-65fe-48f5-bb51-858789b1679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:17.020631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc994c2d-65fe-48f5-bb51-858789b1679f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:17.119235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed823e7-d7cd-48d6-9bb5-17a661a7a4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:17.120237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed823e7-d7cd-48d6-9bb5-17a661a7a4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:17.120750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed823e7-d7cd-48d6-9bb5-17a661a7a4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:17.121221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed823e7-d7cd-48d6-9bb5-17a661a7a4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:17.123454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9ed823e7-d7cd-48d6-9bb5-17a661a7a4d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:17.189252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:17.410522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed823e7-d7cd-48d6-9bb5-17a661a7a4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:17.410522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed823e7-d7cd-48d6-9bb5-17a661a7a4d2 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:17.410522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 653/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:19.604874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707fff1-a0b0-4d1f-840d-758cd0703dc7 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:19.605454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707fff1-a0b0-4d1f-840d-758cd0703dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:19.605454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707fff1-a0b0-4d1f-840d-758cd0703dc7 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:19.606969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 659/1312] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:19.612065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2edf54-6762-4d66-9dbf-8b4a3e355bd5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:19.612536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2edf54-6762-4d66-9dbf-8b4a3e355bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:19.612677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2edf54-6762-4d66-9dbf-8b4a3e355bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:19.689534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8c2edf54-6762-4d66-9dbf-8b4a3e355bd5 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:19.690772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8c2edf54-6762-4d66-9dbf-8b4a3e355bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:19.816100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 654/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:19.855263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707fff1-a0b0-4d1f-840d-758cd0703dc7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:19.856939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4707fff1-a0b0-4d1f-840d-758cd0703dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:19.984655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707fff1-a0b0-4d1f-840d-758cd0703dc7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:20.060082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:20.062330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cca940-9c22-4c05-9eb3-6be7d684132f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:20.062988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cca940-9c22-4c05-9eb3-6be7d684132f service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:20.063270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cca940-9c22-4c05-9eb3-6be7d684132f service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:20.063544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cca940-9c22-4c05-9eb3-6be7d684132f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:20.065208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-49cca940-9c22-4c05-9eb3-6be7d684132f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:20.181812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cca940-9c22-4c05-9eb3-6be7d684132f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:20.183175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cca940-9c22-4c05-9eb3-6be7d684132f service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:20.201856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 660/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:22.772274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0cc10-2b83-4561-ba53-71ec5f112595 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:22.772873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0cc10-2b83-4561-ba53-71ec5f112595 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:22.772873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0cc10-2b83-4561-ba53-71ec5f112595 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:22.775127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 655/1315] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:22.780156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805f05-3f9f-41de-b809-2d4351c7d891 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:22.780587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805f05-3f9f-41de-b809-2d4351c7d891 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:22.780751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53805f05-3f9f-41de-b809-2d4351c7d891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:22.850672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-53805f05-3f9f-41de-b809-2d4351c7d891 None None] MFA Rules not processed for user `ec6d14cf644c4cd6b666c86129fcc505`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:22.853217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-53805f05-3f9f-41de-b809-2d4351c7d891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:22.909587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:22.953327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 661/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:22.992769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0cc10-2b83-4561-ba53-71ec5f112595 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:22.994269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ddf0cc10-2b83-4561-ba53-71ec5f112595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:23.003896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:23.096922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0cc10-2b83-4561-ba53-71ec5f112595 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:23.172838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42392b-251d-45d0-b063-0ec21987bbde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:23.173498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42392b-251d-45d0-b063-0ec21987bbde service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:23.173789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42392b-251d-45d0-b063-0ec21987bbde service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:23.174068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42392b-251d-45d0-b063-0ec21987bbde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:23.175750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8b42392b-251d-45d0-b063-0ec21987bbde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:23.243590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:23.292274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42392b-251d-45d0-b063-0ec21987bbde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:23.293774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42392b-251d-45d0-b063-0ec21987bbde service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:23.313766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 656/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:26.208169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917f52b-14a6-44ba-9290-5104b9c2dc2f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:26.208725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917f52b-14a6-44ba-9290-5104b9c2dc2f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:26.208818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917f52b-14a6-44ba-9290-5104b9c2dc2f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:26.210157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 662/1318] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:26.215841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c7f49-013b-4ca1-99f7-856abacb1691 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:26.216243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c7f49-013b-4ca1-99f7-856abacb1691 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:26.216395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c7f49-013b-4ca1-99f7-856abacb1691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:26.280124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-996c7f49-013b-4ca1-99f7-856abacb1691 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:26.281364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-996c7f49-013b-4ca1-99f7-856abacb1691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:26.374499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 657/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:26.413204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917f52b-14a6-44ba-9290-5104b9c2dc2f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:26.414786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4917f52b-14a6-44ba-9290-5104b9c2dc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:26.516639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917f52b-14a6-44ba-9290-5104b9c2dc2f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:26.587715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242cdfc-3432-43d0-af60-f0eef93bf01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:26.588357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242cdfc-3432-43d0-af60-f0eef93bf01a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:26.588663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242cdfc-3432-43d0-af60-f0eef93bf01a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:26.588964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242cdfc-3432-43d0-af60-f0eef93bf01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:26.590592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f242cdfc-3432-43d0-af60-f0eef93bf01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:26.597834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:26.698377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242cdfc-3432-43d0-af60-f0eef93bf01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:26.699849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242cdfc-3432-43d0-af60-f0eef93bf01a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:26.717753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 663/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:29.742974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc013dd8-9597-4107-bbde-b76564c89732 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:29.743533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc013dd8-9597-4107-bbde-b76564c89732 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:29.743627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc013dd8-9597-4107-bbde-b76564c89732 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:29.745207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 658/1321] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:29.750362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d750d-c220-4743-a70f-f209f44abb1b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:29.750749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d750d-c220-4743-a70f-f209f44abb1b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:29.750925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d750d-c220-4743-a70f-f209f44abb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:29.823142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-647d750d-c220-4743-a70f-f209f44abb1b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:29.824363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-647d750d-c220-4743-a70f-f209f44abb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:29.886112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:29.926391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 664/1322] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:29.963150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc013dd8-9597-4107-bbde-b76564c89732 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:29.964811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc013dd8-9597-4107-bbde-b76564c89732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:29.984124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:30.073192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc013dd8-9597-4107-bbde-b76564c89732 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:30.163644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeac5c-1b33-4aea-9272-e5bfaaa7fbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:30.164307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeac5c-1b33-4aea-9272-e5bfaaa7fbaf service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:30.164615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeac5c-1b33-4aea-9272-e5bfaaa7fbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:30.164916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeac5c-1b33-4aea-9272-e5bfaaa7fbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:30.166761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7baeac5c-1b33-4aea-9272-e5bfaaa7fbaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:30.248433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:30.289332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baeac5c-1b33-4aea-9272-e5bfaaa7fbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:30.290941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baeac5c-1b33-4aea-9272-e5bfaaa7fbaf service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:30.311604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 659/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:32.426456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db802-a75d-4e74-b8a8-ab43b2132f6d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:32.427011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db802-a75d-4e74-b8a8-ab43b2132f6d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:32.427109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db802-a75d-4e74-b8a8-ab43b2132f6d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:32.428429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 665/1324] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:32.433764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad830a98-c498-439a-8bf9-249899f57b35 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:32.434216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad830a98-c498-439a-8bf9-249899f57b35 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:32.434325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad830a98-c498-439a-8bf9-249899f57b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:32.500918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ad830a98-c498-439a-8bf9-249899f57b35 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:32.502048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ad830a98-c498-439a-8bf9-249899f57b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:32.601550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 660/1325] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:32.637986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3db802-a75d-4e74-b8a8-ab43b2132f6d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:32.639549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8e3db802-a75d-4e74-b8a8-ab43b2132f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:32.743216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3db802-a75d-4e74-b8a8-ab43b2132f6d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:32.818739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851970bf-9ab3-49cf-80b2-61957cf6da00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:32.819392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851970bf-9ab3-49cf-80b2-61957cf6da00 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:32.819678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851970bf-9ab3-49cf-80b2-61957cf6da00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:32.819953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851970bf-9ab3-49cf-80b2-61957cf6da00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:32.822181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-851970bf-9ab3-49cf-80b2-61957cf6da00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:32.839414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:32.935588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851970bf-9ab3-49cf-80b2-61957cf6da00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:32.937104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851970bf-9ab3-49cf-80b2-61957cf6da00 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:32.956138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 666/1326] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:35.049075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da3422-d69b-4955-ad8f-252a98e1252b None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:35.049571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da3422-d69b-4955-ad8f-252a98e1252b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:35.049664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da3422-d69b-4955-ad8f-252a98e1252b None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:35.051028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 661/1327] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:35.056387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a0fe2-8835-4178-a639-d1022806850b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:35.056906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a0fe2-8835-4178-a639-d1022806850b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:35.057001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a0fe2-8835-4178-a639-d1022806850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:35.127325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b70a0fe2-8835-4178-a639-d1022806850b None None] MFA Rules not processed for user `ec6d14cf644c4cd6b666c86129fcc505`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:35.128581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b70a0fe2-8835-4178-a639-d1022806850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:35.197222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:35.231705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 667/1328] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:38:35.287887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2586d35-bda6-4d1f-b51d-90c0d975dfce None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:35.288229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2586d35-bda6-4d1f-b51d-90c0d975dfce None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:35.288455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2586d35-bda6-4d1f-b51d-90c0d975dfce None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:35.289937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 662/1329] 10.222.0.57 () {54 vars in 964 bytes} [Fri Jul 15 17:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:35.296693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012649d-fba0-4dc2-9fac-950da7909e23 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:35.297142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012649d-fba0-4dc2-9fac-950da7909e23 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:35.297432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012649d-fba0-4dc2-9fac-950da7909e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:35.378981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f012649d-fba0-4dc2-9fac-950da7909e23 None None] MFA Rules not processed for user `35d3862fe9e846f18d8bc3a0fa5e993a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:35.380274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f012649d-fba0-4dc2-9fac-950da7909e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:35.458607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 668/1330] 10.222.0.57 () {58 vars in 1061 bytes} [Fri Jul 15 17:38:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:35.468105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2586d35-bda6-4d1f-b51d-90c0d975dfce None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:35.469660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b2586d35-bda6-4d1f-b51d-90c0d975dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:35.500694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:35.573602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2586d35-bda6-4d1f-b51d-90c0d975dfce None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:35.648852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0260d-9630-4788-8eca-5c0dd0700a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:35.650056 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0260d-9630-4788-8eca-5c0dd0700a3c service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:35.650335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0260d-9630-4788-8eca-5c0dd0700a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:35.650618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0260d-9630-4788-8eca-5c0dd0700a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:35.652238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7db0260d-9630-4788-8eca-5c0dd0700a3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:35.727793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:35.772318 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0260d-9630-4788-8eca-5c0dd0700a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:35.773866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0260d-9630-4788-8eca-5c0dd0700a3c service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:35.793736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 663/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:38.633151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c7ba-d2ef-464a-bac3-285d881dd70f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:38.634194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c7ba-d2ef-464a-bac3-285d881dd70f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:38.634194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501c7ba-d2ef-464a-bac3-285d881dd70f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:38.635639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 669/1332] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:38.641222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447429a2-3b74-4614-80e5-42186c205190 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:38.641750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447429a2-3b74-4614-80e5-42186c205190 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:38.641750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447429a2-3b74-4614-80e5-42186c205190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:38.705288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-447429a2-3b74-4614-80e5-42186c205190 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:38.706508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-447429a2-3b74-4614-80e5-42186c205190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:38.802059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 664/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:38.838353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501c7ba-d2ef-464a-bac3-285d881dd70f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:38.839927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b501c7ba-d2ef-464a-bac3-285d881dd70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:38.909555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:38.942541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501c7ba-d2ef-464a-bac3-285d881dd70f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:39.018772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27d809-3689-4cde-b915-183b8e55f5c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:39.019413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27d809-3689-4cde-b915-183b8e55f5c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:39.019696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27d809-3689-4cde-b915-183b8e55f5c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:39.019968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27d809-3689-4cde-b915-183b8e55f5c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:39.021628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-de27d809-3689-4cde-b915-183b8e55f5c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:39.128034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27d809-3689-4cde-b915-183b8e55f5c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:39.129506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27d809-3689-4cde-b915-183b8e55f5c0 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:39.139921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:39.148225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 670/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:41.225814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb2a0b-b733-43fc-9bf0-a0497700a30a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:41.226339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb2a0b-b733-43fc-9bf0-a0497700a30a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:41.226498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb2a0b-b733-43fc-9bf0-a0497700a30a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:41.228568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 665/1335] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:41.233706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f7ba-efbc-48e4-9d32-0a319896afd6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:41.234122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f7ba-efbc-48e4-9d32-0a319896afd6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:41.234307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f7ba-efbc-48e4-9d32-0a319896afd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:41.298985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7d97f7ba-efbc-48e4-9d32-0a319896afd6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:41.300201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7d97f7ba-efbc-48e4-9d32-0a319896afd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:41.397674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 671/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:41.437977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb2a0b-b733-43fc-9bf0-a0497700a30a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:41.439521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2afb2a0b-b733-43fc-9bf0-a0497700a30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:41.479196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:41.544850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb2a0b-b733-43fc-9bf0-a0497700a30a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:41.615060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f16345-a4cb-41c6-98a5-73f0e4f26d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:41.615694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f16345-a4cb-41c6-98a5-73f0e4f26d94 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:41.615975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f16345-a4cb-41c6-98a5-73f0e4f26d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:41.616246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f16345-a4cb-41c6-98a5-73f0e4f26d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:41.617987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b4f16345-a4cb-41c6-98a5-73f0e4f26d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:41.698243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:41.723759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f16345-a4cb-41c6-98a5-73f0e4f26d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:41.725227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f16345-a4cb-41c6-98a5-73f0e4f26d94 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:41.742700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 666/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:44.481571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b8ef3-e9fc-4123-96f2-15b2b303ac91 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:44.482097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b8ef3-e9fc-4123-96f2-15b2b303ac91 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:44.482194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b8ef3-e9fc-4123-96f2-15b2b303ac91 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:44.483513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 672/1338] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:44.488633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca97cb-43d3-4bd8-9cec-cd4ea2183ac6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:44.489063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca97cb-43d3-4bd8-9cec-cd4ea2183ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:44.489190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca97cb-43d3-4bd8-9cec-cd4ea2183ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:44.552237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-00ca97cb-43d3-4bd8-9cec-cd4ea2183ac6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:44.553461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-00ca97cb-43d3-4bd8-9cec-cd4ea2183ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:44.647904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 667/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:44.684915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b8ef3-e9fc-4123-96f2-15b2b303ac91 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:44.686438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d17b8ef3-e9fc-4123-96f2-15b2b303ac91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:44.761194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:44.788359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b8ef3-e9fc-4123-96f2-15b2b303ac91 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:44.861156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ea39d-3a8a-439a-aab2-45c800f5956b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:44.861766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ea39d-3a8a-439a-aab2-45c800f5956b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:44.862053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ea39d-3a8a-439a-aab2-45c800f5956b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:44.862333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ea39d-3a8a-439a-aab2-45c800f5956b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:44.863995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8e3ea39d-3a8a-439a-aab2-45c800f5956b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:44.975765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ea39d-3a8a-439a-aab2-45c800f5956b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:44.977748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ea39d-3a8a-439a-aab2-45c800f5956b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:44.996642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 673/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:49.303794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17397526-e173-4c52-a7ac-dece612400ed None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:49.304375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17397526-e173-4c52-a7ac-dece612400ed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:49.304375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17397526-e173-4c52-a7ac-dece612400ed None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:49.306535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 668/1341] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:49.311802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf65998-ff87-469c-83dc-ff931cbfc72d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:49.312312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf65998-ff87-469c-83dc-ff931cbfc72d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:49.312413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf65998-ff87-469c-83dc-ff931cbfc72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:49.317414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:49.384328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6bf65998-ff87-469c-83dc-ff931cbfc72d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:49.386445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6bf65998-ff87-469c-83dc-ff931cbfc72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:49.495016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 674/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:38:49.534770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17397526-e173-4c52-a7ac-dece612400ed None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:49.536422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-17397526-e173-4c52-a7ac-dece612400ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:49.586073 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:49.645167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17397526-e173-4c52-a7ac-dece612400ed None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:49.717718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef626be-afd7-4f05-ab1a-3de78ea987c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:49.718371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef626be-afd7-4f05-ab1a-3de78ea987c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:49.718652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef626be-afd7-4f05-ab1a-3de78ea987c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:49.718930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef626be-afd7-4f05-ab1a-3de78ea987c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:49.720559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1ef626be-afd7-4f05-ab1a-3de78ea987c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:49.814371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:49.830277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef626be-afd7-4f05-ab1a-3de78ea987c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:49.831744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef626be-afd7-4f05-ab1a-3de78ea987c8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:49.851058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 669/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:52.837501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ce511-754a-4bb9-be8f-6ee7c8be0410 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:52.838066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ce511-754a-4bb9-be8f-6ee7c8be0410 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:52.838066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ce511-754a-4bb9-be8f-6ee7c8be0410 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:52.839518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 675/1344] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:52.844901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1446f49-bf92-4d45-889a-310c632b4808 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:52.845308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1446f49-bf92-4d45-889a-310c632b4808 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:52.845474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1446f49-bf92-4d45-889a-310c632b4808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:52.909146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c1446f49-bf92-4d45-889a-310c632b4808 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:52.910353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c1446f49-bf92-4d45-889a-310c632b4808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:53.009661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 670/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:38:53.049903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ce511-754a-4bb9-be8f-6ee7c8be0410 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:53.051536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e16ce511-754a-4bb9-be8f-6ee7c8be0410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:53.142734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:53.160375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ce511-754a-4bb9-be8f-6ee7c8be0410 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:53.235633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7982a-6041-4ea6-93f5-6add9db0d993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:53.236292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7982a-6041-4ea6-93f5-6add9db0d993 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:53.236576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7982a-6041-4ea6-93f5-6add9db0d993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:53.238158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7982a-6041-4ea6-93f5-6add9db0d993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:53.238533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-68f7982a-6041-4ea6-93f5-6add9db0d993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:53.353426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7982a-6041-4ea6-93f5-6add9db0d993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:53.354921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7982a-6041-4ea6-93f5-6add9db0d993 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:53.375867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 676/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:38:58.052206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e700f5-d660-4b5b-a095-e1264c42aa08 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:58.052784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e700f5-d660-4b5b-a095-e1264c42aa08 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:58.052784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e700f5-d660-4b5b-a095-e1264c42aa08 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:58.055203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 671/1347] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:38:58.060228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d3d7c-f418-4e0f-907e-003b12dbeeed None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:58.060786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d3d7c-f418-4e0f-907e-003b12dbeeed None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:58.060893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d3d7c-f418-4e0f-907e-003b12dbeeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:58.076481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:58.133808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a97d3d7c-f418-4e0f-907e-003b12dbeeed None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:38:58.135057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a97d3d7c-f418-4e0f-907e-003b12dbeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:58.235659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 677/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:38:58.274561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e700f5-d660-4b5b-a095-e1264c42aa08 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:38:58.276200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e9e700f5-d660-4b5b-a095-e1264c42aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:58.347659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:58.396281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e700f5-d660-4b5b-a095-e1264c42aa08 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:38:58.474448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086266d4-e8da-4a74-a3e4-47db9295e6db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:38:58.475086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086266d4-e8da-4a74-a3e4-47db9295e6db service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:38:58.475369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086266d4-e8da-4a74-a3e4-47db9295e6db service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:38:58.475680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086266d4-e8da-4a74-a3e4-47db9295e6db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:38:58.477907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-086266d4-e8da-4a74-a3e4-47db9295e6db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:38:58.584732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:38:58.588382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086266d4-e8da-4a74-a3e4-47db9295e6db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:38:58.589875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086266d4-e8da-4a74-a3e4-47db9295e6db service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:38:58.612777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 672/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:03.589772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca83fd-b17d-48cd-85af-4f84148023c1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:03.590591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca83fd-b17d-48cd-85af-4f84148023c1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:03.591094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca83fd-b17d-48cd-85af-4f84148023c1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:03.593139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 678/1350] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:03.598736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183722-c015-4111-bcf0-fe9a4473811f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:03.599127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183722-c015-4111-bcf0-fe9a4473811f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:03.599364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183722-c015-4111-bcf0-fe9a4473811f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:03.753652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1a183722-c015-4111-bcf0-fe9a4473811f None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:03.754853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1a183722-c015-4111-bcf0-fe9a4473811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:03.877343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 673/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:39:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:08.082015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b8b4-c946-41ba-9e1c-7a37ecf64021 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:08.082556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b8b4-c946-41ba-9e1c-7a37ecf64021 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:08.082556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b8b4-c946-41ba-9e1c-7a37ecf64021 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:08.083948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 679/1352] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:08.089767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fceedf-5197-4730-a811-594fbad2e8b6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:08.090190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fceedf-5197-4730-a811-594fbad2e8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:08.090462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fceedf-5197-4730-a811-594fbad2e8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:08.147745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:08.171305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-46fceedf-5197-4730-a811-594fbad2e8b6 None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:08.172613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-46fceedf-5197-4730-a811-594fbad2e8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:08.233685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=59 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:08.295851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 674/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:39:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:09.532648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d0460-f80b-40c4-bcfd-302417d2b17f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:09.533185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d0460-f80b-40c4-bcfd-302417d2b17f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:09.533185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d0460-f80b-40c4-bcfd-302417d2b17f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:09.534495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 680/1354] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:09.540076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ccb6e-d9a6-49a1-a9cf-0b7f043baf98 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:09.540520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ccb6e-d9a6-49a1-a9cf-0b7f043baf98 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:09.540653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ccb6e-d9a6-49a1-a9cf-0b7f043baf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:09.614704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2c6ccb6e-d9a6-49a1-a9cf-0b7f043baf98 None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:09.616021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2c6ccb6e-d9a6-49a1-a9cf-0b7f043baf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:09.658704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:09.724714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 675/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:39:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:39:13.305953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055068b-eac2-4376-a8bd-ba6e82e1b966 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:13.306917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055068b-eac2-4376-a8bd-ba6e82e1b966 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:13.307833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055068b-eac2-4376-a8bd-ba6e82e1b966 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:13.309960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 681/1356] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:13.315607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690b8a5-e2b8-4ac9-9bf6-4d348e1ddc53 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:13.316474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690b8a5-e2b8-4ac9-9bf6-4d348e1ddc53 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:13.317017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690b8a5-e2b8-4ac9-9bf6-4d348e1ddc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:13.391072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a690b8a5-e2b8-4ac9-9bf6-4d348e1ddc53 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:13.396958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a690b8a5-e2b8-4ac9-9bf6-4d348e1ddc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:13.499166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 676/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:13.574229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ab0f-064b-4580-b497-a35c55c99ae0 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:13.574861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ab0f-064b-4580-b497-a35c55c99ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:13.575526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926ab0f-064b-4580-b497-a35c55c99ae0 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:13.577946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 682/1358] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 15 17:39:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:13.587651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1878-3f73-4c80-8fd6-8406527e86d5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:13.592323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1878-3f73-4c80-8fd6-8406527e86d5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:13.592323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1878-3f73-4c80-8fd6-8406527e86d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:13.605490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:13.669331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-267a1878-3f73-4c80-8fd6-8406527e86d5 None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:13.670825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-267a1878-3f73-4c80-8fd6-8406527e86d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:13.794933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 677/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 15 17:39:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:13.802665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ab0f-064b-4580-b497-a35c55c99ae0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:13.804904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4926ab0f-064b-4580-b497-a35c55c99ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:13.958268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926ab0f-064b-4580-b497-a35c55c99ae0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:14.037638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:14.083888 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4da06-21bf-41c4-8f27-06836132cbc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:14.084639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4da06-21bf-41c4-8f27-06836132cbc1 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:14.085006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4da06-21bf-41c4-8f27-06836132cbc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:14.085353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4da06-21bf-41c4-8f27-06836132cbc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:14.087393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ccc4da06-21bf-41c4-8f27-06836132cbc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:14.186104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4da06-21bf-41c4-8f27-06836132cbc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:14.187820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4da06-21bf-41c4-8f27-06836132cbc1 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:14.207502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 683/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:16.437237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd38be-6ff5-4630-9a8a-c288a82208f8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:16.437830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd38be-6ff5-4630-9a8a-c288a82208f8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:16.437999 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd38be-6ff5-4630-9a8a-c288a82208f8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:16.440783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 678/1361] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:16.448544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2718-9d5d-40ff-b60a-b2a9245a719f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:16.449244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2718-9d5d-40ff-b60a-b2a9245a719f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:16.449244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba2718-9d5d-40ff-b60a-b2a9245a719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:16.523436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bdba2718-9d5d-40ff-b60a-b2a9245a719f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:16.524585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bdba2718-9d5d-40ff-b60a-b2a9245a719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:16.564086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:16.628097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 684/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:16.672825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd38be-6ff5-4630-9a8a-c288a82208f8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:16.674290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6fbd38be-6ff5-4630-9a8a-c288a82208f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:16.754807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:16.849664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd38be-6ff5-4630-9a8a-c288a82208f8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:17.004447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ab1bd-8d3c-438f-ae79-4b37d52ae209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:17.009335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab1bd-8d3c-438f-ae79-4b37d52ae209 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:17.009335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab1bd-8d3c-438f-ae79-4b37d52ae209 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:17.009335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab1bd-8d3c-438f-ae79-4b37d52ae209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:17.009335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-da2ab1bd-8d3c-438f-ae79-4b37d52ae209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:17.046645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:17.121573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ab1bd-8d3c-438f-ae79-4b37d52ae209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:17.123181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ab1bd-8d3c-438f-ae79-4b37d52ae209 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:17.143832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 679/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:19.669390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848c11-b76b-422b-9044-89e8ab1d7818 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:19.670265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848c11-b76b-422b-9044-89e8ab1d7818 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:19.670265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848c11-b76b-422b-9044-89e8ab1d7818 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:19.671522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 685/1364] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:19.678390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7644f-e524-445b-8907-e3787b5b6fbc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:19.678865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7644f-e524-445b-8907-e3787b5b6fbc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:19.679888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7644f-e524-445b-8907-e3787b5b6fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:19.797114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8cf7644f-e524-445b-8907-e3787b5b6fbc None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:19.799188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8cf7644f-e524-445b-8907-e3787b5b6fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:19.900969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:19.908647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 680/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:19.955796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6144d45-b0d7-4ab0-a88a-803c259327e9 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:19.956361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6144d45-b0d7-4ab0-a88a-803c259327e9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:19.956868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6144d45-b0d7-4ab0-a88a-803c259327e9 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:19.958732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 686/1366] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 15 17:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:19.963860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d06bfa-b631-4410-b083-1c25088bf3ac None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:19.964210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d06bfa-b631-4410-b083-1c25088bf3ac None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:19.964495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d06bfa-b631-4410-b083-1c25088bf3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:20.049902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-52d06bfa-b631-4410-b083-1c25088bf3ac None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:20.051217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-52d06bfa-b631-4410-b083-1c25088bf3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:20.192760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 681/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 15 17:39:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:39:20.201035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6144d45-b0d7-4ab0-a88a-803c259327e9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:20.202571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a6144d45-b0d7-4ab0-a88a-803c259327e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:20.367538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6144d45-b0d7-4ab0-a88a-803c259327e9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:20.389801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:20.494450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2227aa-e690-4bc7-b546-bfda07000284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:20.495108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2227aa-e690-4bc7-b546-bfda07000284 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:20.495461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2227aa-e690-4bc7-b546-bfda07000284 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:20.495802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2227aa-e690-4bc7-b546-bfda07000284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:20.498222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0d2227aa-e690-4bc7-b546-bfda07000284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:20.602377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2227aa-e690-4bc7-b546-bfda07000284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:20.604101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2227aa-e690-4bc7-b546-bfda07000284 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:20.626999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 687/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:21.553701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbae8a-f4f0-4458-80e5-c8e78c1b99be None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:21.554296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbae8a-f4f0-4458-80e5-c8e78c1b99be None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:21.554405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbae8a-f4f0-4458-80e5-c8e78c1b99be None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:21.555890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 682/1369] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:21.560848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcfc2f-f783-421d-a5aa-75dda2b7fd96 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:21.561287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcfc2f-f783-421d-a5aa-75dda2b7fd96 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:21.561486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcfc2f-f783-421d-a5aa-75dda2b7fd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:21.596283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:21.631846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-53dcfc2f-f783-421d-a5aa-75dda2b7fd96 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:21.633183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-53dcfc2f-f783-421d-a5aa-75dda2b7fd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:21.742844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 688/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:22.084661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434cada-0b5e-4d84-b238-1b969f82d21c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:22.084867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434cada-0b5e-4d84-b238-1b969f82d21c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:22.085660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434cada-0b5e-4d84-b238-1b969f82d21c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:22.087204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 683/1371] 10.222.0.57 () {54 vars in 963 bytes} [Fri Jul 15 17:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:22.091947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3fe6-a57e-4176-a2e9-4770989cea2e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:22.092239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3fe6-a57e-4176-a2e9-4770989cea2e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:22.092519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea3fe6-a57e-4176-a2e9-4770989cea2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:22.170059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c7ea3fe6-a57e-4176-a2e9-4770989cea2e None None] MFA Rules not processed for user `4c1a2fa417e1456ba4c36d6e4dc4014d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:22.171270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c7ea3fe6-a57e-4176-a2e9-4770989cea2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:22.221800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:22.258468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 689/1372] 10.222.0.57 () {58 vars in 1060 bytes} [Fri Jul 15 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:22.266698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434cada-0b5e-4d84-b238-1b969f82d21c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:22.268331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e434cada-0b5e-4d84-b238-1b969f82d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:22.324555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:22.380136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434cada-0b5e-4d84-b238-1b969f82d21c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:22.458309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec41cd-62da-4716-b426-8de8f76bcee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:22.458999 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec41cd-62da-4716-b426-8de8f76bcee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:22.459268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec41cd-62da-4716-b426-8de8f76bcee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:22.459547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec41cd-62da-4716-b426-8de8f76bcee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:22.461283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-35ec41cd-62da-4716-b426-8de8f76bcee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:22.577846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:22.591541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec41cd-62da-4716-b426-8de8f76bcee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:22.593043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec41cd-62da-4716-b426-8de8f76bcee2 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:22.614407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 684/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:39:22.858199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b0e19-ff95-465c-9ded-150b9685ed3c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:22.858671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b0e19-ff95-465c-9ded-150b9685ed3c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:22.858671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b0e19-ff95-465c-9ded-150b9685ed3c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:22.863803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 690/1374] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:22.865518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064c70-b2e8-4767-a39e-8899d9c4d8c7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:22.865661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064c70-b2e8-4767-a39e-8899d9c4d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:22.866037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064c70-b2e8-4767-a39e-8899d9c4d8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:22.938868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-58064c70-b2e8-4767-a39e-8899d9c4d8c7 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:22.940093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-58064c70-b2e8-4767-a39e-8899d9c4d8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:23.042123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 685/1375] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:23.086842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b0e19-ff95-465c-9ded-150b9685ed3c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:23.088304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e78b0e19-ff95-465c-9ded-150b9685ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:23.241107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b0e19-ff95-465c-9ded-150b9685ed3c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:23.295658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:23.359692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b198889-5f21-45d1-8522-5408b3605838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:23.360257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b198889-5f21-45d1-8522-5408b3605838 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:23.360643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b198889-5f21-45d1-8522-5408b3605838 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:23.360900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b198889-5f21-45d1-8522-5408b3605838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:23.362617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0b198889-5f21-45d1-8522-5408b3605838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:23.452797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b198889-5f21-45d1-8522-5408b3605838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:23.454388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b198889-5f21-45d1-8522-5408b3605838 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:23.472734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 691/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:39:25.612678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe760e-531a-40f9-b20a-e2bac457f734 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:25.613181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe760e-531a-40f9-b20a-e2bac457f734 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:25.613274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe760e-531a-40f9-b20a-e2bac457f734 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:25.614629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 686/1377] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:25.620397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb234305-36da-4194-a983-0600f15f2386 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:25.620853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb234305-36da-4194-a983-0600f15f2386 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:25.621024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb234305-36da-4194-a983-0600f15f2386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:25.687425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fb234305-36da-4194-a983-0600f15f2386 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:25.688652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fb234305-36da-4194-a983-0600f15f2386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:25.697340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:25.812657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 692/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:25.859674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe760e-531a-40f9-b20a-e2bac457f734 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:25.861404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a1fe760e-531a-40f9-b20a-e2bac457f734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:25.929379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:26.016217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe760e-531a-40f9-b20a-e2bac457f734 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:26.133599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40986ff-722c-416e-b59d-9859e1cf951b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:26.134256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40986ff-722c-416e-b59d-9859e1cf951b service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:26.134548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40986ff-722c-416e-b59d-9859e1cf951b service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:26.134882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40986ff-722c-416e-b59d-9859e1cf951b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:26.136552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f40986ff-722c-416e-b59d-9859e1cf951b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:26.157513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:26.258897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40986ff-722c-416e-b59d-9859e1cf951b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:26.260867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40986ff-722c-416e-b59d-9859e1cf951b service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:26.287200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 687/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:28.461972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982a9e-104a-4823-9a39-e8f38f7d65a4 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:28.462496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982a9e-104a-4823-9a39-e8f38f7d65a4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:28.462496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982a9e-104a-4823-9a39-e8f38f7d65a4 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:28.463962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 693/1380] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:28.469259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd2354-0f94-4575-aa3b-a4ea8de343c7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:28.469708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd2354-0f94-4575-aa3b-a4ea8de343c7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:28.469821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd2354-0f94-4575-aa3b-a4ea8de343c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:28.544328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-20dd2354-0f94-4575-aa3b-a4ea8de343c7 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:28.545619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-20dd2354-0f94-4575-aa3b-a4ea8de343c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:28.645855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:28.673067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 688/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:28.705347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982a9e-104a-4823-9a39-e8f38f7d65a4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:28.706876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d8982a9e-104a-4823-9a39-e8f38f7d65a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:28.853085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:28.864290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982a9e-104a-4823-9a39-e8f38f7d65a4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:28.986819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a950fd2-8e42-4815-b090-c9a7000459dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:28.987387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950fd2-8e42-4815-b090-c9a7000459dd service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:28.987796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950fd2-8e42-4815-b090-c9a7000459dd service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:28.988085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a950fd2-8e42-4815-b090-c9a7000459dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:28.989775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9a950fd2-8e42-4815-b090-c9a7000459dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:29.096015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a950fd2-8e42-4815-b090-c9a7000459dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:29.097678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a950fd2-8e42-4815-b090-c9a7000459dd service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:29.109654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:29.120773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 694/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:31.352337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b05fd8-260b-4d82-a1cf-7598648f19aa None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:31.352914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b05fd8-260b-4d82-a1cf-7598648f19aa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:31.353077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b05fd8-260b-4d82-a1cf-7598648f19aa None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:31.354870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 689/1383] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:31.360297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e81dda-b686-4512-9871-6e97f4dd72c5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:31.360757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e81dda-b686-4512-9871-6e97f4dd72c5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:31.360895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e81dda-b686-4512-9871-6e97f4dd72c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:31.430018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a9e81dda-b686-4512-9871-6e97f4dd72c5 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:31.431259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a9e81dda-b686-4512-9871-6e97f4dd72c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:31.538013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 695/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:31.581089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b05fd8-260b-4d82-a1cf-7598648f19aa None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:31.582681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-14b05fd8-260b-4d82-a1cf-7598648f19aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:31.737655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b05fd8-260b-4d82-a1cf-7598648f19aa None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:31.805345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:31.854700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e65b6-077e-4170-a436-c9271190e524 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:31.855360 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e65b6-077e-4170-a436-c9271190e524 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:31.855647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e65b6-077e-4170-a436-c9271190e524 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:31.855929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e65b6-077e-4170-a436-c9271190e524 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:31.857633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cd9e65b6-077e-4170-a436-c9271190e524 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:31.964139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e65b6-077e-4170-a436-c9271190e524 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:31.965791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e65b6-077e-4170-a436-c9271190e524 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:31.984165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 690/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:34.490062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d09035-d4bd-4e6e-9471-e7a2fb27c73e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:34.490698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d09035-d4bd-4e6e-9471-e7a2fb27c73e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:34.490698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d09035-d4bd-4e6e-9471-e7a2fb27c73e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:34.492522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 696/1386] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:39:34.497877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78719fdd-c747-4cb4-8066-4747a52ffdfc None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:34.498461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78719fdd-c747-4cb4-8066-4747a52ffdfc None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:34.498461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78719fdd-c747-4cb4-8066-4747a52ffdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:34.568249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-78719fdd-c747-4cb4-8066-4747a52ffdfc None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:34.570198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-78719fdd-c747-4cb4-8066-4747a52ffdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:34.603695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:34.680436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 691/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:34.736825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d09035-d4bd-4e6e-9471-e7a2fb27c73e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:34.738453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-16d09035-d4bd-4e6e-9471-e7a2fb27c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:34.831974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:34.906233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d09035-d4bd-4e6e-9471-e7a2fb27c73e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:35.033036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5d2d-2988-4a0a-8a52-5a4167678250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:35.034090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5d2d-2988-4a0a-8a52-5a4167678250 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:35.034448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5d2d-2988-4a0a-8a52-5a4167678250 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:35.034857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5d2d-2988-4a0a-8a52-5a4167678250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:35.037001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8b0d5d2d-2988-4a0a-8a52-5a4167678250 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:35.086416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:35.151190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d5d2d-2988-4a0a-8a52-5a4167678250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:35.153312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d5d2d-2988-4a0a-8a52-5a4167678250 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:35.176174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 697/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:37.402025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae885e-bd27-4d9d-afe0-4a5d51a1bb02 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:37.402574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae885e-bd27-4d9d-afe0-4a5d51a1bb02 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:37.402574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae885e-bd27-4d9d-afe0-4a5d51a1bb02 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:37.403955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 692/1389] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:37.409202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee211f-8d83-49a1-8cd5-df80f2088188 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:37.409809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee211f-8d83-49a1-8cd5-df80f2088188 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:37.409809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee211f-8d83-49a1-8cd5-df80f2088188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:37.476206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-50ee211f-8d83-49a1-8cd5-df80f2088188 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:37.477448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-50ee211f-8d83-49a1-8cd5-df80f2088188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:37.575768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 698/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:37.619444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae885e-bd27-4d9d-afe0-4a5d51a1bb02 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:37.621060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-93ae885e-bd27-4d9d-afe0-4a5d51a1bb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:37.767979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae885e-bd27-4d9d-afe0-4a5d51a1bb02 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:37.784862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:37.880309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73963d6d-f8a6-42a3-9915-8a8c02ce530b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:37.880904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73963d6d-f8a6-42a3-9915-8a8c02ce530b service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:37.881194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73963d6d-f8a6-42a3-9915-8a8c02ce530b service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:37.882004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73963d6d-f8a6-42a3-9915-8a8c02ce530b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:37.883642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-73963d6d-f8a6-42a3-9915-8a8c02ce530b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:37.972713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73963d6d-f8a6-42a3-9915-8a8c02ce530b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:37.974367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73963d6d-f8a6-42a3-9915-8a8c02ce530b service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:37.992492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 693/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:37.998711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b5d34-c6cc-4e1f-865e-780b7fca3921 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:37.999019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b5d34-c6cc-4e1f-865e-780b7fca3921 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:37.999307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b5d34-c6cc-4e1f-865e-780b7fca3921 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:38.000809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 699/1392] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:39:38.006003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8f3f8-8e0d-4eea-b059-7fe3f1e0fb4c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:38.010259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8f3f8-8e0d-4eea-b059-7fe3f1e0fb4c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:38.010259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8f3f8-8e0d-4eea-b059-7fe3f1e0fb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:38.032408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:38.072937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-afd8f3f8-8e0d-4eea-b059-7fe3f1e0fb4c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:38.074173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-afd8f3f8-8e0d-4eea-b059-7fe3f1e0fb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:38.172458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 694/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:39:38.231226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295af64c-a6f9-4a5c-bd8c-9ca46284b168 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:38.231517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295af64c-a6f9-4a5c-bd8c-9ca46284b168 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:38.231903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295af64c-a6f9-4a5c-bd8c-9ca46284b168 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:38.233453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 700/1394] 10.222.0.57 () {54 vars in 960 bytes} [Fri Jul 15 17:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:39:38.239541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca3711-0bc7-4b78-a72e-a2911cc47819 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:38.239928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca3711-0bc7-4b78-a72e-a2911cc47819 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:38.240285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca3711-0bc7-4b78-a72e-a2911cc47819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:38.326127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8cca3711-0bc7-4b78-a72e-a2911cc47819 None None] MFA Rules not processed for user `b3b0f154abf64d57a431b956a1822300`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:38.327375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8cca3711-0bc7-4b78-a72e-a2911cc47819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:38.372190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:38.409177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 695/1395] 10.222.0.57 () {58 vars in 1057 bytes} [Fri Jul 15 17:39:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:38.416962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295af64c-a6f9-4a5c-bd8c-9ca46284b168 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:38.418460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-295af64c-a6f9-4a5c-bd8c-9ca46284b168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:38.432495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:38.525592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295af64c-a6f9-4a5c-bd8c-9ca46284b168 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:38.600202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c1ec1-4669-45ab-88d8-1b7ef79d5126 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:38.601466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1ec1-4669-45ab-88d8-1b7ef79d5126 service glance] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:38.601754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1ec1-4669-45ab-88d8-1b7ef79d5126 service glance] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:38.602032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1ec1-4669-45ab-88d8-1b7ef79d5126 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:38.603684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-427c1ec1-4669-45ab-88d8-1b7ef79d5126 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:38.674000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:38.717072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c1ec1-4669-45ab-88d8-1b7ef79d5126 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:38.718433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c1ec1-4669-45ab-88d8-1b7ef79d5126 service glance] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:38.736186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 701/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:39.607766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5871-230a-4e31-bfbc-7ce4760b8336 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:39.608273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5871-230a-4e31-bfbc-7ce4760b8336 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:39.609286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5871-230a-4e31-bfbc-7ce4760b8336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:39.674951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a56d5871-230a-4e31-bfbc-7ce4760b8336 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:39.676166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a56d5871-230a-4e31-bfbc-7ce4760b8336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:39.775204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 696/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:39:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:39.830773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837469f6-b125-46ef-a406-59b1f385a30b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:39.831332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837469f6-b125-46ef-a406-59b1f385a30b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:39.831458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837469f6-b125-46ef-a406-59b1f385a30b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:39.833991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 702/1398] 10.222.0.57 () {54 vars in 942 bytes} [Fri Jul 15 17:39:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:39.843322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5d153-e37c-4cfa-b722-083733aa9965 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:39.843668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5d153-e37c-4cfa-b722-083733aa9965 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:39.843991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5d153-e37c-4cfa-b722-083733aa9965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:39.916585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:39.924233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-63d5d153-e37c-4cfa-b722-083733aa9965 None None] MFA Rules not processed for user `7a49603273fe4eff892e8b024a473158`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:39.930354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-63d5d153-e37c-4cfa-b722-083733aa9965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:40.034387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 697/1399] 10.222.0.57 () {58 vars in 1039 bytes} [Fri Jul 15 17:39:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:39:40.044389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837469f6-b125-46ef-a406-59b1f385a30b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:40.045921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-837469f6-b125-46ef-a406-59b1f385a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:40.182683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837469f6-b125-46ef-a406-59b1f385a30b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:40.211777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242a2c-1f93-454a-84f7-6385141d44a2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:40.212064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242a2c-1f93-454a-84f7-6385141d44a2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:40.212365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242a2c-1f93-454a-84f7-6385141d44a2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:40.213813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 698/1400] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:39:40.218745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:40.219059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:40.219340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:40.220620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:40.286644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:40.288302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:40.294023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e006d-c29b-4ff9-bc94-af7a37390385 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:40.294688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e006d-c29b-4ff9-bc94-af7a37390385 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:40.294975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e006d-c29b-4ff9-bc94-af7a37390385 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:40.295257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e006d-c29b-4ff9-bc94-af7a37390385 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:40.296955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-912e006d-c29b-4ff9-bc94-af7a37390385 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:40.349905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:40.381201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e006d-c29b-4ff9-bc94-af7a37390385 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:40.383384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e006d-c29b-4ff9-bc94-af7a37390385 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:40.387041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 699/1401] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:39:40.392038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 703/1402] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:39:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:39:40.432792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:40.434894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:40.546988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a60c7-a0b7-4b44-a2dc-3fe3213a32fb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:40.620791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:40.663956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdcf27-25a6-4f55-97da-a757cafc9df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:40.664633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcf27-25a6-4f55-97da-a757cafc9df0 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:40.664914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcf27-25a6-4f55-97da-a757cafc9df0 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:40.665189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcf27-25a6-4f55-97da-a757cafc9df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:40.666816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-31cdcf27-25a6-4f55-97da-a757cafc9df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:40.757442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdcf27-25a6-4f55-97da-a757cafc9df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:40.759213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdcf27-25a6-4f55-97da-a757cafc9df0 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:40.787742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 700/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:42.994565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dd8a4-8af9-419a-a4f8-36f433e19e1a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:42.995486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dd8a4-8af9-419a-a4f8-36f433e19e1a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:42.995486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dd8a4-8af9-419a-a4f8-36f433e19e1a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:42.996682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 704/1404] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:43.002089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca601-c65f-47e1-9763-f8ed28fdb3f8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:43.002510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca601-c65f-47e1-9763-f8ed28fdb3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:43.002703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca601-c65f-47e1-9763-f8ed28fdb3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:43.090793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-df9ca601-c65f-47e1-9763-f8ed28fdb3f8 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:43.092572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-df9ca601-c65f-47e1-9763-f8ed28fdb3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:43.146547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:43.247765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 701/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:39:43.301115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dd8a4-8af9-419a-a4f8-36f433e19e1a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:43.302933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cf6dd8a4-8af9-419a-a4f8-36f433e19e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:43.437657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:43.551817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dd8a4-8af9-419a-a4f8-36f433e19e1a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:43.709478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd58d4-358f-49cd-9098-88dccfd57a19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:43.710140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd58d4-358f-49cd-9098-88dccfd57a19 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:43.710429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd58d4-358f-49cd-9098-88dccfd57a19 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:43.711974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd58d4-358f-49cd-9098-88dccfd57a19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:43.714545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6bbd58d4-358f-49cd-9098-88dccfd57a19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:43.846907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd58d4-358f-49cd-9098-88dccfd57a19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:43.848506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd58d4-358f-49cd-9098-88dccfd57a19 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:43.870910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 705/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:44.043881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:46.177961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884f972-739e-4702-9213-1ef9d012e36a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:46.178591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884f972-739e-4702-9213-1ef9d012e36a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:46.178591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884f972-739e-4702-9213-1ef9d012e36a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:46.183774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 702/1407] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:39:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:39:46.185282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c7ad5-d030-4ab1-b8d1-3e6816f9df32 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:46.185711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c7ad5-d030-4ab1-b8d1-3e6816f9df32 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:46.185869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c7ad5-d030-4ab1-b8d1-3e6816f9df32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:46.263904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-035c7ad5-d030-4ab1-b8d1-3e6816f9df32 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:39:46.265674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-035c7ad5-d030-4ab1-b8d1-3e6816f9df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:46.384151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 706/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:39:46.422066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884f972-739e-4702-9213-1ef9d012e36a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:39:46.423591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1884f972-739e-4702-9213-1ef9d012e36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:46.588871 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884f972-739e-4702-9213-1ef9d012e36a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:39:46.613011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:46.720394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd53b65-1d55-476d-a4b9-e86df3a238fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:39:46.721212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53b65-1d55-476d-a4b9-e86df3a238fb service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:39:46.721565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53b65-1d55-476d-a4b9-e86df3a238fb service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:39:46.721914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd53b65-1d55-476d-a4b9-e86df3a238fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:39:46.723662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3dd53b65-1d55-476d-a4b9-e86df3a238fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:39:46.823903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd53b65-1d55-476d-a4b9-e86df3a238fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:39:46.825676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd53b65-1d55-476d-a4b9-e86df3a238fb service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:39:46.849469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 703/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:39:56.374925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:39:56.845101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:04.375646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07eefe-a619-4872-95ee-2a4cc6446d6c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:04.376125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07eefe-a619-4872-95ee-2a4cc6446d6c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:04.376236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07eefe-a619-4872-95ee-2a4cc6446d6c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:04.377689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 707/1410] 10.222.0.57 () {54 vars in 918 bytes} [Fri Jul 15 17:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:04.382924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cf8a2-5d40-48dd-bc69-c57841df5893 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:04.383590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cf8a2-5d40-48dd-bc69-c57841df5893 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:04.383875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cf8a2-5d40-48dd-bc69-c57841df5893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:04.458164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-df8cf8a2-5d40-48dd-bc69-c57841df5893 None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:04.459422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-df8cf8a2-5d40-48dd-bc69-c57841df5893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:04.564064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 704/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Jul 15 17:40:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:04.578133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853e883-6859-4ad9-98f9-902ce6add88b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:04.578429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853e883-6859-4ad9-98f9-902ce6add88b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:04.578711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853e883-6859-4ad9-98f9-902ce6add88b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:04.580093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 708/1412] 10.222.0.57 () {54 vars in 918 bytes} [Fri Jul 15 17:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:04.585388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3553a-b212-401d-a0a0-43d98e20b476 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:04.585683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3553a-b212-401d-a0a0-43d98e20b476 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:04.585971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3553a-b212-401d-a0a0-43d98e20b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:04.656790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:04.659423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-73c3553a-b212-401d-a0a0-43d98e20b476 None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:04.660692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-73c3553a-b212-401d-a0a0-43d98e20b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:04.775927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 705/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Jul 15 17:40:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:07.254005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb292b-a057-4ba5-a49c-9a537d45ef76 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:07.254483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb292b-a057-4ba5-a49c-9a537d45ef76 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:07.254593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb292b-a057-4ba5-a49c-9a537d45ef76 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:07.256490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:07.261631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e3208-efad-4813-8091-f58ecb82458e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:07.262038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e3208-efad-4813-8091-f58ecb82458e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:07.262200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e3208-efad-4813-8091-f58ecb82458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:07.326335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b91e3208-efad-4813-8091-f58ecb82458e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:07.327565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b91e3208-efad-4813-8091-f58ecb82458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:07.382944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:07.429217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:07.472352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb292b-a057-4ba5-a49c-9a537d45ef76 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:07.473911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-75fb292b-a057-4ba5-a49c-9a537d45ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:07.622073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb292b-a057-4ba5-a49c-9a537d45ef76 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:07.695593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:07.736442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc29c6-60e7-4229-b403-ea9865b7c82d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:07.737122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc29c6-60e7-4229-b403-ea9865b7c82d service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:07.737407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc29c6-60e7-4229-b403-ea9865b7c82d service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:07.737692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc29c6-60e7-4229-b403-ea9865b7c82d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:07.739336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5cfc29c6-60e7-4229-b403-ea9865b7c82d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:07.828477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc29c6-60e7-4229-b403-ea9865b7c82d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:07.830160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc29c6-60e7-4229-b403-ea9865b7c82d service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:07.848561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 710/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:08.744690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905ab6-44ca-4870-8a28-75c2b5e3e6a7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:08.745257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905ab6-44ca-4870-8a28-75c2b5e3e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:08.745257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905ab6-44ca-4870-8a28-75c2b5e3e6a7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:08.746626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:08.751539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4c812-f222-4e64-83d1-2361bdd3a44f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:08.752144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4c812-f222-4e64-83d1-2361bdd3a44f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:08.752144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4c812-f222-4e64-83d1-2361bdd3a44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:08.831556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-aac4c812-f222-4e64-83d1-2361bdd3a44f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:08.832876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aac4c812-f222-4e64-83d1-2361bdd3a44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:08.859738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:08.929099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:40:09.077633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4905ab6-44ca-4870-8a28-75c2b5e3e6a7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:09.079138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a4905ab6-44ca-4870-8a28-75c2b5e3e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:09.222396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4905ab6-44ca-4870-8a28-75c2b5e3e6a7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:09.263337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:09.335236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ceec12-4189-4cf8-a04c-6aae97ff2bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:09.335811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceec12-4189-4cf8-a04c-6aae97ff2bbc service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:09.336098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceec12-4189-4cf8-a04c-6aae97ff2bbc service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:09.336374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceec12-4189-4cf8-a04c-6aae97ff2bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:09.338042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f6ceec12-4189-4cf8-a04c-6aae97ff2bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:09.427388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ceec12-4189-4cf8-a04c-6aae97ff2bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:09.429110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ceec12-4189-4cf8-a04c-6aae97ff2bbc service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:09.447647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 708/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:10.333407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac76cc-3c5a-4337-a4e5-dccfad8d6f78 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:10.333923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac76cc-3c5a-4337-a4e5-dccfad8d6f78 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:10.334044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac76cc-3c5a-4337-a4e5-dccfad8d6f78 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:10.335406 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:10.340444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46be38c-9192-4204-a05f-f12a293faa91 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:10.340873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46be38c-9192-4204-a05f-f12a293faa91 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:10.341048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46be38c-9192-4204-a05f-f12a293faa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:10.393804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:10.415262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d46be38c-9192-4204-a05f-f12a293faa91 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:10.416470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d46be38c-9192-4204-a05f-f12a293faa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:10.514000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:10.664266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac76cc-3c5a-4337-a4e5-dccfad8d6f78 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:10.665830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4dac76cc-3c5a-4337-a4e5-dccfad8d6f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:10.812933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac76cc-3c5a-4337-a4e5-dccfad8d6f78 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:10.825521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:10.925350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ffbe-1993-430e-a8f3-40fa808ae0e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:10.926018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ffbe-1993-430e-a8f3-40fa808ae0e3 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:10.926285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ffbe-1993-430e-a8f3-40fa808ae0e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:10.926562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ffbe-1993-430e-a8f3-40fa808ae0e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:10.928795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d257ffbe-1993-430e-a8f3-40fa808ae0e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:11.017603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257ffbe-1993-430e-a8f3-40fa808ae0e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:11.019549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257ffbe-1993-430e-a8f3-40fa808ae0e3 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:11.038922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 713/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:11.892126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477dba0-0f03-4f59-8560-6bf9e1cc11a7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:11.892733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477dba0-0f03-4f59-8560-6bf9e1cc11a7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:11.892733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477dba0-0f03-4f59-8560-6bf9e1cc11a7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:11.894106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:11.899705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab03bba-9558-4be0-83c7-da0bd219924e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:11.900125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab03bba-9558-4be0-83c7-da0bd219924e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:11.900310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab03bba-9558-4be0-83c7-da0bd219924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:11.918393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:11.973270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1ab03bba-9558-4be0-83c7-da0bd219924e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:11.974470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1ab03bba-9558-4be0-83c7-da0bd219924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:12.076145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:12.224815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477dba0-0f03-4f59-8560-6bf9e1cc11a7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:12.226351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4477dba0-0f03-4f59-8560-6bf9e1cc11a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:12.335779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:12.373259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477dba0-0f03-4f59-8560-6bf9e1cc11a7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:12.484150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5237977-1f49-415c-afd7-b9741953db66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:12.484755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5237977-1f49-415c-afd7-b9741953db66 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:12.485043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5237977-1f49-415c-afd7-b9741953db66 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:12.485320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5237977-1f49-415c-afd7-b9741953db66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:12.486960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5237977-1f49-415c-afd7-b9741953db66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:12.561485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:12.575044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5237977-1f49-415c-afd7-b9741953db66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:12.576693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5237977-1f49-415c-afd7-b9741953db66 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:12.595095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 711/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:13.459584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf912101-de4a-4f65-bb5a-7c124a0b16b8 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:13.459584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf912101-de4a-4f65-bb5a-7c124a0b16b8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:13.460160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf912101-de4a-4f65-bb5a-7c124a0b16b8 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:13.461322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:13.466367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45dcb4-36e8-48aa-83c7-54e48c76023a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:13.466804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45dcb4-36e8-48aa-83c7-54e48c76023a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:13.466940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45dcb4-36e8-48aa-83c7-54e48c76023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:13.541396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0b45dcb4-36e8-48aa-83c7-54e48c76023a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:13.542579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0b45dcb4-36e8-48aa-83c7-54e48c76023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:13.637843 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:13.788233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf912101-de4a-4f65-bb5a-7c124a0b16b8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:13.789786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bf912101-de4a-4f65-bb5a-7c124a0b16b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:13.873307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:13.936962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf912101-de4a-4f65-bb5a-7c124a0b16b8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:14.054771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720b449-42d3-4cb1-8ab2-0dee36954ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:14.058216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720b449-42d3-4cb1-8ab2-0dee36954ed4 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:14.058216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720b449-42d3-4cb1-8ab2-0dee36954ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:14.058216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720b449-42d3-4cb1-8ab2-0dee36954ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:14.058216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0720b449-42d3-4cb1-8ab2-0dee36954ed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:14.119721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:14.160723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720b449-42d3-4cb1-8ab2-0dee36954ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:14.162798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720b449-42d3-4cb1-8ab2-0dee36954ed4 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:14.182391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 716/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:15.046595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa484a95-d644-41de-b660-e7252f5e8ef9 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:15.047145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa484a95-d644-41de-b660-e7252f5e8ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:15.047145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa484a95-d644-41de-b660-e7252f5e8ef9 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:15.048638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:15.054218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8283c-d919-4533-a9cc-513c85f84304 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:15.054603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8283c-d919-4533-a9cc-513c85f84304 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:15.054779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8283c-d919-4533-a9cc-513c85f84304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:15.129916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0fd8283c-d919-4533-a9cc-513c85f84304 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:15.131153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0fd8283c-d919-4533-a9cc-513c85f84304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:15.231328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:15.379588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa484a95-d644-41de-b660-e7252f5e8ef9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:15.381658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aa484a95-d644-41de-b660-e7252f5e8ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:15.430005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:15.529590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa484a95-d644-41de-b660-e7252f5e8ef9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:15.643068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3e448-ac63-4063-8e3a-ac83e97d1b2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:15.643721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e448-ac63-4063-8e3a-ac83e97d1b2d service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:15.644010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e448-ac63-4063-8e3a-ac83e97d1b2d service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:15.644294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3e448-ac63-4063-8e3a-ac83e97d1b2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:15.646007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16f3e448-ac63-4063-8e3a-ac83e97d1b2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:15.652873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:15.737327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3e448-ac63-4063-8e3a-ac83e97d1b2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:15.738868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3e448-ac63-4063-8e3a-ac83e97d1b2d service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:15.757064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 714/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:16.609481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a718-24e3-4f16-b92d-6ed8e2c0482f None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:16.610035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a718-24e3-4f16-b92d-6ed8e2c0482f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:16.610035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a718-24e3-4f16-b92d-6ed8e2c0482f None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:16.611444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:16.616772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb404bb-0a75-442c-ba2c-4bdba7e2a6eb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:16.617178 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb404bb-0a75-442c-ba2c-4bdba7e2a6eb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:16.617338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb404bb-0a75-442c-ba2c-4bdba7e2a6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:16.690866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bbb404bb-0a75-442c-ba2c-4bdba7e2a6eb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:16.692080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbb404bb-0a75-442c-ba2c-4bdba7e2a6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:16.758674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:16.787433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:16.938677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a718-24e3-4f16-b92d-6ed8e2c0482f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:16.940161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e940a718-24e3-4f16-b92d-6ed8e2c0482f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:16.958163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:17.089206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a718-24e3-4f16-b92d-6ed8e2c0482f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:17.176429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:17.202743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b097c-4362-47d8-a1a4-303e734856e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:17.203335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b097c-4362-47d8-a1a4-303e734856e7 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:17.203619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b097c-4362-47d8-a1a4-303e734856e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:17.203896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b097c-4362-47d8-a1a4-303e734856e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:17.205640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ff8b097c-4362-47d8-a1a4-303e734856e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:17.296439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b097c-4362-47d8-a1a4-303e734856e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:17.298145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b097c-4362-47d8-a1a4-303e734856e7 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:17.317197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 719/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:18.178498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ab05-0553-4ed5-9e21-89bef08daaca None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:18.179040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ab05-0553-4ed5-9e21-89bef08daaca None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:18.179040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ab05-0553-4ed5-9e21-89bef08daaca None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:18.180456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:18.185514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec514951-eaf4-4bdc-8f1e-426e07642287 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:18.185989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec514951-eaf4-4bdc-8f1e-426e07642287 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:18.186445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec514951-eaf4-4bdc-8f1e-426e07642287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:18.269159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ec514951-eaf4-4bdc-8f1e-426e07642287 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:18.270384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ec514951-eaf4-4bdc-8f1e-426e07642287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:18.312252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:18.375942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:18.525661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1ab05-0553-4ed5-9e21-89bef08daaca None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:18.527106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56f1ab05-0553-4ed5-9e21-89bef08daaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:18.680957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1ab05-0553-4ed5-9e21-89bef08daaca None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:18.735483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:18.794942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb4710-0275-431a-9368-7afb412bf179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:18.795516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4710-0275-431a-9368-7afb412bf179 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:18.795802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4710-0275-431a-9368-7afb412bf179 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:18.796082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4710-0275-431a-9368-7afb412bf179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:18.797759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6beb4710-0275-431a-9368-7afb412bf179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:18.887141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb4710-0275-431a-9368-7afb412bf179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:18.889059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb4710-0275-431a-9368-7afb412bf179 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:18.907073 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 717/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:19.792088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcbdf9-247d-44fb-a2b2-7e424b84558a None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:19.792687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcbdf9-247d-44fb-a2b2-7e424b84558a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:19.792687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcbdf9-247d-44fb-a2b2-7e424b84558a None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:19.793991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:19.799012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe6a5b-1e7c-416a-873d-1a97dad1bd57 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:19.799454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe6a5b-1e7c-416a-873d-1a97dad1bd57 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:19.799577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe6a5b-1e7c-416a-873d-1a97dad1bd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:19.874277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d1fe6a5b-1e7c-416a-873d-1a97dad1bd57 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:19.875498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d1fe6a5b-1e7c-416a-873d-1a97dad1bd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:19.877570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:19.974984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:20.137712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcbdf9-247d-44fb-a2b2-7e424b84558a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:20.139498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-77fcbdf9-247d-44fb-a2b2-7e424b84558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:20.285988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcbdf9-247d-44fb-a2b2-7e424b84558a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:20.312496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:20.399173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21ac69-44ae-4220-a09d-b8455c9b9d7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:20.399759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ac69-44ae-4220-a09d-b8455c9b9d7e service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:20.400043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ac69-44ae-4220-a09d-b8455c9b9d7e service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:20.400322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ac69-44ae-4220-a09d-b8455c9b9d7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:20.401990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9d21ac69-44ae-4220-a09d-b8455c9b9d7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:20.490445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21ac69-44ae-4220-a09d-b8455c9b9d7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:20.492198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21ac69-44ae-4220-a09d-b8455c9b9d7e service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:20.509905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 722/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:22.458985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30255-b79a-4084-84d4-023cdd206450 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:22.459534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30255-b79a-4084-84d4-023cdd206450 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:22.459534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d30255-b79a-4084-84d4-023cdd206450 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:22.460948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:22.466151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f91df8-5183-4db9-9a2a-abdc47bc15c6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:22.466600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f91df8-5183-4db9-9a2a-abdc47bc15c6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:22.466709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f91df8-5183-4db9-9a2a-abdc47bc15c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:22.507054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:22.532411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-21f91df8-5183-4db9-9a2a-abdc47bc15c6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:22.534415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-21f91df8-5183-4db9-9a2a-abdc47bc15c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:22.632799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:22.671072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d30255-b79a-4084-84d4-023cdd206450 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:22.672670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5d30255-b79a-4084-84d4-023cdd206450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:22.774259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d30255-b79a-4084-84d4-023cdd206450 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:22.805426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:22.846290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a049e3-ba67-4fce-901a-5c00a8ffcf75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:22.846941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a049e3-ba67-4fce-901a-5c00a8ffcf75 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:22.847225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a049e3-ba67-4fce-901a-5c00a8ffcf75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:22.847503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a049e3-ba67-4fce-901a-5c00a8ffcf75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:22.849957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-04a049e3-ba67-4fce-901a-5c00a8ffcf75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:22.963265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a049e3-ba67-4fce-901a-5c00a8ffcf75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:22.964774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a049e3-ba67-4fce-901a-5c00a8ffcf75 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:22.983243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 720/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:25.137048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542b9e0-82d6-4e81-886b-b2651ddb99c0 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:25.137593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542b9e0-82d6-4e81-886b-b2651ddb99c0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:25.137593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542b9e0-82d6-4e81-886b-b2651ddb99c0 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:25.139263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:25.144675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504a33-4c21-486b-962a-305142b7e153 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:25.145123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504a33-4c21-486b-962a-305142b7e153 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:25.145229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a504a33-4c21-486b-962a-305142b7e153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:25.211891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8a504a33-4c21-486b-962a-305142b7e153 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:25.213166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8a504a33-4c21-486b-962a-305142b7e153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:25.226002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:25.309817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:25.347085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542b9e0-82d6-4e81-886b-b2651ddb99c0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:25.349209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1542b9e0-82d6-4e81-886b-b2651ddb99c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:25.451680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542b9e0-82d6-4e81-886b-b2651ddb99c0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:25.464224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:25.525717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcdeea-6698-4e2b-8b87-900bb7e0b032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:25.526376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcdeea-6698-4e2b-8b87-900bb7e0b032 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:25.526665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcdeea-6698-4e2b-8b87-900bb7e0b032 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:25.526944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcdeea-6698-4e2b-8b87-900bb7e0b032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:25.528636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cbdcdeea-6698-4e2b-8b87-900bb7e0b032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:25.637671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcdeea-6698-4e2b-8b87-900bb7e0b032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:25.639364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcdeea-6698-4e2b-8b87-900bb7e0b032 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:25.656963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 725/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:28.032994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e6941-3ce9-49dd-8dc9-2774f403eef0 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:28.033530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e6941-3ce9-49dd-8dc9-2774f403eef0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:28.033530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e6941-3ce9-49dd-8dc9-2774f403eef0 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:28.035023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:28.041028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c311e-1208-471a-9cd8-d36553ce8c61 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:28.041458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c311e-1208-471a-9cd8-d36553ce8c61 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:28.041594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c311e-1208-471a-9cd8-d36553ce8c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:28.090311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:28.109148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7c6c311e-1208-471a-9cd8-d36553ce8c61 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:28.111033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7c6c311e-1208-471a-9cd8-d36553ce8c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:28.207531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:28.246314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e6941-3ce9-49dd-8dc9-2774f403eef0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:28.247852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1b2e6941-3ce9-49dd-8dc9-2774f403eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:28.349863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e6941-3ce9-49dd-8dc9-2774f403eef0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:28.390179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:28.423973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ae4a7-f94e-4bf9-b834-da6f8fe6e7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:28.424641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ae4a7-f94e-4bf9-b834-da6f8fe6e7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:28.424933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ae4a7-f94e-4bf9-b834-da6f8fe6e7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:28.425213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ae4a7-f94e-4bf9-b834-da6f8fe6e7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:28.427384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ce5ae4a7-f94e-4bf9-b834-da6f8fe6e7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:28.533954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ae4a7-f94e-4bf9-b834-da6f8fe6e7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:28.535437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ae4a7-f94e-4bf9-b834-da6f8fe6e7a5 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:28.555008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 723/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:30.688050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd41fbc-328b-4482-97fc-9273e1c077b2 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:30.688586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd41fbc-328b-4482-97fc-9273e1c077b2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:30.688586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd41fbc-328b-4482-97fc-9273e1c077b2 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:30.690015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:30.695799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84948c6-3e77-44f7-a9e2-8f9cd32ce036 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:30.696207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84948c6-3e77-44f7-a9e2-8f9cd32ce036 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:30.696371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84948c6-3e77-44f7-a9e2-8f9cd32ce036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:30.760142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f84948c6-3e77-44f7-a9e2-8f9cd32ce036 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:30.761428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f84948c6-3e77-44f7-a9e2-8f9cd32ce036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:30.781589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:30.856212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:30.895916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd41fbc-328b-4482-97fc-9273e1c077b2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:30.897504 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6fd41fbc-328b-4482-97fc-9273e1c077b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:31.002473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd41fbc-328b-4482-97fc-9273e1c077b2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:31.023188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:31.076164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac6f64-b2e4-4af8-a0a1-e95ebec364f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:31.076855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6f64-b2e4-4af8-a0a1-e95ebec364f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:31.077147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6f64-b2e4-4af8-a0a1-e95ebec364f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:31.077427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6f64-b2e4-4af8-a0a1-e95ebec364f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:31.079096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-78ac6f64-b2e4-4af8-a0a1-e95ebec364f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:31.189427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac6f64-b2e4-4af8-a0a1-e95ebec364f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:31.190877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac6f64-b2e4-4af8-a0a1-e95ebec364f3 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:31.209805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 728/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:33.276109 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06331ba2-97cf-4f51-b1e9-44fdbd5de882 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:33.276648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06331ba2-97cf-4f51-b1e9-44fdbd5de882 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:33.276776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06331ba2-97cf-4f51-b1e9-44fdbd5de882 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:33.278114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:33.283390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c370ad-26fd-48a6-93f8-11568c50afff None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:33.283797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c370ad-26fd-48a6-93f8-11568c50afff None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:33.283960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c370ad-26fd-48a6-93f8-11568c50afff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:33.350847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:33.353599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-03c370ad-26fd-48a6-93f8-11568c50afff None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:33.354819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-03c370ad-26fd-48a6-93f8-11568c50afff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:33.451391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:33.489907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06331ba2-97cf-4f51-b1e9-44fdbd5de882 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:33.491440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-06331ba2-97cf-4f51-b1e9-44fdbd5de882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:33.611903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06331ba2-97cf-4f51-b1e9-44fdbd5de882 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:33.659774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:33.684458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b250-3135-454e-a509-6d319633019f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:33.686961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b250-3135-454e-a509-6d319633019f service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:33.686961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b250-3135-454e-a509-6d319633019f service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:33.686961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b250-3135-454e-a509-6d319633019f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:33.687378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b280b250-3135-454e-a509-6d319633019f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:33.792882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b250-3135-454e-a509-6d319633019f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:33.794324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b250-3135-454e-a509-6d319633019f service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:33.812536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 726/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:35.889076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebab339-a51a-4cc5-8d1d-0eafc83935df None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:35.889653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebab339-a51a-4cc5-8d1d-0eafc83935df None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:35.890671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebab339-a51a-4cc5-8d1d-0eafc83935df None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:35.892093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:35.897630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5626f06-4924-488b-af30-3470b0fa8c41 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:35.898062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5626f06-4924-488b-af30-3470b0fa8c41 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:35.898201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5626f06-4924-488b-af30-3470b0fa8c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:35.963749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d5626f06-4924-488b-af30-3470b0fa8c41 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:35.965000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d5626f06-4924-488b-af30-3470b0fa8c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:35.994588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:36.064298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:36.105590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebab339-a51a-4cc5-8d1d-0eafc83935df None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:36.107195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5ebab339-a51a-4cc5-8d1d-0eafc83935df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:36.221219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebab339-a51a-4cc5-8d1d-0eafc83935df None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:36.256429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:36.306324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad948b05-043f-4e52-8bc6-8f72d519b22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:36.306970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948b05-043f-4e52-8bc6-8f72d519b22b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:36.307253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948b05-043f-4e52-8bc6-8f72d519b22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:36.307531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948b05-043f-4e52-8bc6-8f72d519b22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:36.309520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ad948b05-043f-4e52-8bc6-8f72d519b22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:36.421743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad948b05-043f-4e52-8bc6-8f72d519b22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:36.423188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad948b05-043f-4e52-8bc6-8f72d519b22b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:36.441359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 731/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:38.559689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c104-aae7-4eaf-a986-db21e9db79d8 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:38.560211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c104-aae7-4eaf-a986-db21e9db79d8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:38.560331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c104-aae7-4eaf-a986-db21e9db79d8 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:38.561694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:38.567733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49f83c-b4d8-4148-ba47-3101237c1e73 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:38.568204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49f83c-b4d8-4148-ba47-3101237c1e73 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:38.568301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49f83c-b4d8-4148-ba47-3101237c1e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:38.633476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4f49f83c-b4d8-4148-ba47-3101237c1e73 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:38.634682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4f49f83c-b4d8-4148-ba47-3101237c1e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:38.643099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:38.730254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:38.768562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75c104-aae7-4eaf-a986-db21e9db79d8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:38.770172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0a75c104-aae7-4eaf-a986-db21e9db79d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:38.872521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75c104-aae7-4eaf-a986-db21e9db79d8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:38.926315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:38.944769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac20ba-07a9-49c6-8407-3d4abdab5ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:38.945424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac20ba-07a9-49c6-8407-3d4abdab5ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:38.945710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac20ba-07a9-49c6-8407-3d4abdab5ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:38.945986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac20ba-07a9-49c6-8407-3d4abdab5ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:38.947629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-76ac20ba-07a9-49c6-8407-3d4abdab5ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:39.054301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac20ba-07a9-49c6-8407-3d4abdab5ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:39.055758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac20ba-07a9-49c6-8407-3d4abdab5ea4 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:39.073877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 729/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:41.104701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6180532-6f55-49d9-8bc0-2ec9a4c2f122 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:41.105280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6180532-6f55-49d9-8bc0-2ec9a4c2f122 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:41.105396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6180532-6f55-49d9-8bc0-2ec9a4c2f122 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:41.106702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:41.111879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd27d8-3980-43b7-811b-9434c44ec837 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:41.112442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd27d8-3980-43b7-811b-9434c44ec837 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:41.112547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd27d8-3980-43b7-811b-9434c44ec837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:41.178214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f5bd27d8-3980-43b7-811b-9434c44ec837 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:41.179430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f5bd27d8-3980-43b7-811b-9434c44ec837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:41.216770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:41.276233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:41.313662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6180532-6f55-49d9-8bc0-2ec9a4c2f122 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:41.315257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c6180532-6f55-49d9-8bc0-2ec9a4c2f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:41.418570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6180532-6f55-49d9-8bc0-2ec9a4c2f122 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:41.455341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:41.491596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974648e8-43ad-4403-b988-068b5ce12e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:41.492245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974648e8-43ad-4403-b988-068b5ce12e11 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:41.492535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974648e8-43ad-4403-b988-068b5ce12e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:41.492845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974648e8-43ad-4403-b988-068b5ce12e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:41.494522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-974648e8-43ad-4403-b988-068b5ce12e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:41.602177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974648e8-43ad-4403-b988-068b5ce12e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:41.603617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974648e8-43ad-4403-b988-068b5ce12e11 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:41.621899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 734/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:43.690229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31195ac-2125-47fb-b3fa-805580de7938 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:43.690770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31195ac-2125-47fb-b3fa-805580de7938 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:43.690899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31195ac-2125-47fb-b3fa-805580de7938 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:43.692854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:43.698081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ec9db-dc91-482e-9966-016514f75231 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:43.698513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ec9db-dc91-482e-9966-016514f75231 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:43.698650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ec9db-dc91-482e-9966-016514f75231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:43.761918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bc0ec9db-dc91-482e-9966-016514f75231 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:43.763386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bc0ec9db-dc91-482e-9966-016514f75231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:43.781061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:43.859194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:43.902423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31195ac-2125-47fb-b3fa-805580de7938 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:43.903959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c31195ac-2125-47fb-b3fa-805580de7938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:44.011871 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31195ac-2125-47fb-b3fa-805580de7938 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:44.081424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:44.088947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a521fbc-fabb-40d4-8979-637336d99721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:44.089598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521fbc-fabb-40d4-8979-637336d99721 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:44.090722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521fbc-fabb-40d4-8979-637336d99721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:44.091006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521fbc-fabb-40d4-8979-637336d99721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:44.092700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1a521fbc-fabb-40d4-8979-637336d99721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:44.221054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a521fbc-fabb-40d4-8979-637336d99721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:44.222514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a521fbc-fabb-40d4-8979-637336d99721 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:44.245174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 732/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:46.337935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19bb2b-1c78-4fd9-90cc-1ce39c489a0d None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:46.338500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19bb2b-1c78-4fd9-90cc-1ce39c489a0d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:46.338500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19bb2b-1c78-4fd9-90cc-1ce39c489a0d None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:46.339903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:40:46.345939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b5582-1c57-417e-bbef-d4e10169feae None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:46.346390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b5582-1c57-417e-bbef-d4e10169feae None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:46.346538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b5582-1c57-417e-bbef-d4e10169feae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:46.411585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a78b5582-1c57-417e-bbef-d4e10169feae None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:40:46.412837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a78b5582-1c57-417e-bbef-d4e10169feae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:46.459709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:46.513516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:40:46.551522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19bb2b-1c78-4fd9-90cc-1ce39c489a0d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:40:46.553131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0e19bb2b-1c78-4fd9-90cc-1ce39c489a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:46.655974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19bb2b-1c78-4fd9-90cc-1ce39c489a0d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:40:46.706065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:46.732199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b69e5-9d92-4180-a28d-7f3421f411c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:40:46.732891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b69e5-9d92-4180-a28d-7f3421f411c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:40:46.733183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b69e5-9d92-4180-a28d-7f3421f411c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:40:46.733465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b69e5-9d92-4180-a28d-7f3421f411c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:40:46.735119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7f2b69e5-9d92-4180-a28d-7f3421f411c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:40:46.843448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b69e5-9d92-4180-a28d-7f3421f411c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:40:46.844923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b69e5-9d92-4180-a28d-7f3421f411c6 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:40:46.864215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 737/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:40:56.511357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=22 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:40:56.861345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=66 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:06.388003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d75e6-5118-4b95-b15c-e8862f253135 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:06.388410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d75e6-5118-4b95-b15c-e8862f253135 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:06.388561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d75e6-5118-4b95-b15c-e8862f253135 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:06.391162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 734/1471] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:06.396582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63c0f-0ff3-4c0e-99c8-722e1efdb021 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:06.397072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63c0f-0ff3-4c0e-99c8-722e1efdb021 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:06.397169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d63c0f-0ff3-4c0e-99c8-722e1efdb021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:06.462407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-67d63c0f-0ff3-4c0e-99c8-722e1efdb021 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:06.463619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-67d63c0f-0ff3-4c0e-99c8-722e1efdb021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:06.559806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 738/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:06.607915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99621e27-268d-477b-90ba-0f610cd8ebec None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:06.608205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99621e27-268d-477b-90ba-0f610cd8ebec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:06.608485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99621e27-268d-477b-90ba-0f610cd8ebec None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:06.609897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 735/1473] 10.222.0.57 () {54 vars in 960 bytes} [Fri Jul 15 17:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:06.615528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea238441-2ec6-4a75-9fe7-817d8df892bf None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:06.615824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea238441-2ec6-4a75-9fe7-817d8df892bf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:06.616129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea238441-2ec6-4a75-9fe7-817d8df892bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:06.690068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ea238441-2ec6-4a75-9fe7-817d8df892bf None None] MFA Rules not processed for user `b3b0f154abf64d57a431b956a1822300`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:06.691323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ea238441-2ec6-4a75-9fe7-817d8df892bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:06.712130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:06.767320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 739/1474] 10.222.0.57 () {58 vars in 1057 bytes} [Fri Jul 15 17:41:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:06.775337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99621e27-268d-477b-90ba-0f610cd8ebec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:41:06.776921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-99621e27-268d-477b-90ba-0f610cd8ebec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:06.876545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99621e27-268d-477b-90ba-0f610cd8ebec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:41:06.946726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e3b2e-56a1-42cf-823d-38ff5dbffba1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:41:06.947367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e3b2e-56a1-42cf-823d-38ff5dbffba1 service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:06.947653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e3b2e-56a1-42cf-823d-38ff5dbffba1 service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:06.947928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e3b2e-56a1-42cf-823d-38ff5dbffba1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:06.949580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6e1e3b2e-56a1-42cf-823d-38ff5dbffba1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:07.002269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:07.056860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e3b2e-56a1-42cf-823d-38ff5dbffba1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:41:07.058324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e3b2e-56a1-42cf-823d-38ff5dbffba1 service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:41:07.076589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 736/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:41:09.205103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2bb6e-35c8-4f1a-97dc-457d698add2c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:09.205676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2bb6e-35c8-4f1a-97dc-457d698add2c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:09.205676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2bb6e-35c8-4f1a-97dc-457d698add2c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:09.207585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 740/1476] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:09.212869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90ae3-ab9f-4e1d-a8a0-6c64b45a10ad None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:09.213308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90ae3-ab9f-4e1d-a8a0-6c64b45a10ad None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:09.213442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90ae3-ab9f-4e1d-a8a0-6c64b45a10ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:09.278282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bfe90ae3-ab9f-4e1d-a8a0-6c64b45a10ad None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:09.279510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bfe90ae3-ab9f-4e1d-a8a0-6c64b45a10ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:09.372966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 737/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:09.396466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2bb6e-35c8-4f1a-97dc-457d698add2c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:41:09.398009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-afc2bb6e-35c8-4f1a-97dc-457d698add2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:09.526477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2bb6e-35c8-4f1a-97dc-457d698add2c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:41:09.574584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:09.618596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4bdd1-a080-4dd1-9a44-ce9d5a500c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:41:09.619260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4bdd1-a080-4dd1-9a44-ce9d5a500c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:09.619548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4bdd1-a080-4dd1-9a44-ce9d5a500c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:09.619823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4bdd1-a080-4dd1-9a44-ce9d5a500c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:09.620557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4bdd1-a080-4dd1-9a44-ce9d5a500c65 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:41:09.622381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-52d4bdd1-a080-4dd1-9a44-ce9d5a500c65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:41:09.622381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:41:09.622381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:41:09.623756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4bdd1-a080-4dd1-9a44-ce9d5a500c65 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:41:09.643760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 741/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Jul 15 17:41:09 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:41:11.680367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550e43-bbde-420a-8e2b-2f75cccecf29 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:11.680979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550e43-bbde-420a-8e2b-2f75cccecf29 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:11.680979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df550e43-bbde-420a-8e2b-2f75cccecf29 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:11.682352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 738/1479] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:11.687568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605aa67-4162-4b69-8b60-16ab35f8df24 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:11.688013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605aa67-4162-4b69-8b60-16ab35f8df24 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:11.688121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605aa67-4162-4b69-8b60-16ab35f8df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:11.753697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4605aa67-4162-4b69-8b60-16ab35f8df24 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:11.754911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4605aa67-4162-4b69-8b60-16ab35f8df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:11.848921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 742/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:11.891459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550e43-bbde-420a-8e2b-2f75cccecf29 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:41:11.893055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-df550e43-bbde-420a-8e2b-2f75cccecf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:11.903233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:12.038848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df550e43-bbde-420a-8e2b-2f75cccecf29 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:41:12.118351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:12.151813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324b2f3-d234-4ea3-962d-a2320d5e7190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:41:12.152467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324b2f3-d234-4ea3-962d-a2320d5e7190 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:12.152794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324b2f3-d234-4ea3-962d-a2320d5e7190 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:12.153076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324b2f3-d234-4ea3-962d-a2320d5e7190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:12.154727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0324b2f3-d234-4ea3-962d-a2320d5e7190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:12.243291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324b2f3-d234-4ea3-962d-a2320d5e7190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:41:12.244958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324b2f3-d234-4ea3-962d-a2320d5e7190 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:41:12.263781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 739/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:41:14.451646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149801d2-34cd-478b-9ee1-3bc8f3f8b9c3 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:14.452197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149801d2-34cd-478b-9ee1-3bc8f3f8b9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:14.452197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149801d2-34cd-478b-9ee1-3bc8f3f8b9c3 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:14.453735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 743/1482] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:14.458941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6895f-a4cf-4d6b-956a-1e0099abb02d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:14.459382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6895f-a4cf-4d6b-956a-1e0099abb02d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:14.459535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6895f-a4cf-4d6b-956a-1e0099abb02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:14.524795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-33d6895f-a4cf-4d6b-956a-1e0099abb02d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:14.526011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-33d6895f-a4cf-4d6b-956a-1e0099abb02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:14.571924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:14.622275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 740/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:14.657762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149801d2-34cd-478b-9ee1-3bc8f3f8b9c3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:41:14.659306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-149801d2-34cd-478b-9ee1-3bc8f3f8b9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:14.683673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:14.764185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149801d2-34cd-478b-9ee1-3bc8f3f8b9c3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:41:14.837917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d9a7-c4d4-4519-9ab3-91e1f10fdb3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:41:14.838571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d9a7-c4d4-4519-9ab3-91e1f10fdb3a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:14.838858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d9a7-c4d4-4519-9ab3-91e1f10fdb3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:14.839137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d9a7-c4d4-4519-9ab3-91e1f10fdb3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:14.840824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c3c6d9a7-c4d4-4519-9ab3-91e1f10fdb3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:14.914739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:14.950133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d9a7-c4d4-4519-9ab3-91e1f10fdb3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:41:14.951585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d9a7-c4d4-4519-9ab3-91e1f10fdb3a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:41:14.970087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 744/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:41:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:41:17.051783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d8d8a-dbfd-477d-b8a4-04d2807ee42f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:17.052375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d8d8a-dbfd-477d-b8a4-04d2807ee42f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:17.052375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d8d8a-dbfd-477d-b8a4-04d2807ee42f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:17.053793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 741/1485] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:17.059160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2758e-11a3-421b-bdc0-612ff597ed0f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:17.059553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2758e-11a3-421b-bdc0-612ff597ed0f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:17.059721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2758e-11a3-421b-bdc0-612ff597ed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:17.124996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3ba2758e-11a3-421b-bdc0-612ff597ed0f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:17.126226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3ba2758e-11a3-421b-bdc0-612ff597ed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:17.228228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 745/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:17.266555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d8d8a-dbfd-477d-b8a4-04d2807ee42f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:41:17.268098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a83d8d8a-dbfd-477d-b8a4-04d2807ee42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:17.371650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d8d8a-dbfd-477d-b8a4-04d2807ee42f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:41:17.441918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:17.443668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e399fe5-b5e6-4a20-aad2-9e223519baf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:41:17.444314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e399fe5-b5e6-4a20-aad2-9e223519baf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:17.444625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e399fe5-b5e6-4a20-aad2-9e223519baf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:17.444900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e399fe5-b5e6-4a20-aad2-9e223519baf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:17.446534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7e399fe5-b5e6-4a20-aad2-9e223519baf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:17.555955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e399fe5-b5e6-4a20-aad2-9e223519baf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:41:17.557434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e399fe5-b5e6-4a20-aad2-9e223519baf0 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:41:17.575168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 742/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:41:19.717831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4ed21-6978-4533-8bc3-eac512277af1 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:19.718404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4ed21-6978-4533-8bc3-eac512277af1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:19.718404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4ed21-6978-4533-8bc3-eac512277af1 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:19.719790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 746/1488] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:19.725779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d8de4-0aa2-42e5-8e91-51f919ce8eb5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:19.726182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d8de4-0aa2-42e5-8e91-51f919ce8eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:19.726342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d8de4-0aa2-42e5-8e91-51f919ce8eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:19.790861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e28d8de4-0aa2-42e5-8e91-51f919ce8eb5 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:19.792081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e28d8de4-0aa2-42e5-8e91-51f919ce8eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:19.850680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:19.895535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 743/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:19.935329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4ed21-6978-4533-8bc3-eac512277af1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:41:19.936904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-22a4ed21-6978-4533-8bc3-eac512277af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:19.973872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:20.042598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4ed21-6978-4533-8bc3-eac512277af1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:41:20.116417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360c7ea-a6e5-42a8-a32a-35c1091926d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:41:20.117127 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c7ea-a6e5-42a8-a32a-35c1091926d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:20.117439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c7ea-a6e5-42a8-a32a-35c1091926d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:20.117720 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360c7ea-a6e5-42a8-a32a-35c1091926d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:20.119442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d360c7ea-a6e5-42a8-a32a-35c1091926d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:20.203337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:20.232791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360c7ea-a6e5-42a8-a32a-35c1091926d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:41:20.234212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360c7ea-a6e5-42a8-a32a-35c1091926d7 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:41:20.254811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 747/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:41:22.733866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eaf10-64b4-43bb-b261-fb2f3fcc78ef None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:22.734660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eaf10-64b4-43bb-b261-fb2f3fcc78ef None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:22.734660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eaf10-64b4-43bb-b261-fb2f3fcc78ef None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:22.735881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 744/1491] 10.222.0.57 () {54 vars in 926 bytes} [Fri Jul 15 17:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:41:22.741858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a4f5a-c20f-4b4e-8ccb-642a21037f7e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:22.742315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a4f5a-c20f-4b4e-8ccb-642a21037f7e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:22.742465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a4f5a-c20f-4b4e-8ccb-642a21037f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:22.805979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8e1a4f5a-c20f-4b4e-8ccb-642a21037f7e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:41:22.807173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8e1a4f5a-c20f-4b4e-8ccb-642a21037f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:22.902732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 748/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Fri Jul 15 17:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:41:22.941652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eaf10-64b4-43bb-b261-fb2f3fcc78ef None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:41:22.943186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-331eaf10-64b4-43bb-b261-fb2f3fcc78ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:23.045273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eaf10-64b4-43bb-b261-fb2f3fcc78ef None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:41:23.117997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce4ca4-9a60-4cb2-8a00-4cdd74e3e7d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:41:23.118655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce4ca4-9a60-4cb2-8a00-4cdd74e3e7d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:41:23.118942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce4ca4-9a60-4cb2-8a00-4cdd74e3e7d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:41:23.119227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce4ca4-9a60-4cb2-8a00-4cdd74e3e7d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:41:23.120955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dfce4ca4-9a60-4cb2-8a00-4cdd74e3e7d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:41:23.131660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:23.351901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce4ca4-9a60-4cb2-8a00-4cdd74e3e7d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:41:23.353392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce4ca4-9a60-4cb2-8a00-4cdd74e3e7d5 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:41:23.371551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 745/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:41:32.900914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=82 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:41:33.369237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=50 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:43:20.237417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77b120-2445-4367-8e73-fab46f595ef6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:20.238050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77b120-2445-4367-8e73-fab46f595ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:20.238050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77b120-2445-4367-8e73-fab46f595ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:20.434995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ba77b120-2445-4367-8e73-fab46f595ef6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:43:20.436211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ba77b120-2445-4367-8e73-fab46f595ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:43:20.542677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 749/1494] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:43:20.551319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16484420-4d21-4a6b-b786-536763965a5e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:20.551801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16484420-4d21-4a6b-b786-536763965a5e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:20.551909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16484420-4d21-4a6b-b786-536763965a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:20.569687 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-16484420-4d21-4a6b-b786-536763965a5e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:43:20.571052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16484420-4d21-4a6b-b786-536763965a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:43:20.579885 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 746/1495] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:43:20.584463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77b120-2445-4367-8e73-fab46f595ef6 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:20.594291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77b120-2445-4367-8e73-fab46f595ef6 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:20.599348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:20.599993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:20.600279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:20.600556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:20.601310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:43:20.601615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:20.602616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:20.602616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:43:20.602616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:43:20.603505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:20.615963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 750/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:43:20.623835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16484420-4d21-4a6b-b786-536763965a5e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:20.633133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16484420-4d21-4a6b-b786-536763965a5e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:20.637970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:20.638623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:20.638907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:20.639184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:20.639883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:20.641692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:20.641692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 15 17:43:20.641692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: Jul 15 17:43:20.642887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:20.753208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 747/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:43:20 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:43:20.757384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:20.767665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8052733-561b-4288-8921-5114a81f20ec admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:20.816811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:43:20.823563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:20.827750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:20.827750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:20.827750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:20.827750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:20.827750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:20.827750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:43:20.827750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:43:20.828382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:20.942170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 751/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:43:20 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:43:20.946461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:20.959327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eac88-b49b-446f-91c9-d725ce0ffe33 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:20.965583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:20.966253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:20.966545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:20.966824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:20.967571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:20.968584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:20.968584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 15 17:43:20.968584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: Jul 15 17:43:20.969851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:20.983914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 748/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:43:20.988163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:20.998391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe834647-0fab-4b3a-803e-16d6e99148ef admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:21.002866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063a5ef-d5d2-448a-8678-759ca09c49d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:21.003515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a5ef-d5d2-448a-8678-759ca09c49d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.003792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a5ef-d5d2-448a-8678-759ca09c49d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.004065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063a5ef-d5d2-448a-8678-759ca09c49d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.004842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063a5ef-d5d2-448a-8678-759ca09c49d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:21.005840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-5063a5ef-d5d2-448a-8678-759ca09c49d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:21.005840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:43:21.005840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:43:21.006716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063a5ef-d5d2-448a-8678-759ca09c49d8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:21.020468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 752/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:43:21.024649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:21.035204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b88bb-ce55-4700-8817-31c5104ea06d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:21.039776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904ae91-a87a-4f7f-bec5-a74be4879e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:21.040681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ae91-a87a-4f7f-bec5-a74be4879e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.040995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ae91-a87a-4f7f-bec5-a74be4879e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.041279 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ae91-a87a-4f7f-bec5-a74be4879e7e admin admin] PATH_INFO: `/v3/projects/bac6179235774547a0b9b78af759d72d/users/5147704b537744d5a7b9d80cfc685391/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.055707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904ae91-a87a-4f7f-bec5-a74be4879e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac6179235774547a0b9b78af759d72d, user_id=5147704b537744d5a7b9d80cfc685391, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:21.056782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-a904ae91-a87a-4f7f-bec5-a74be4879e7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:21.056782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 15 17:43:21.056782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: Jul 15 17:43:21.058845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904ae91-a87a-4f7f-bec5-a74be4879e7e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:21.131272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 749/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:43:21 2022] PUT /identity/v3/projects/bac6179235774547a0b9b78af759d72d/users/5147704b537744d5a7b9d80cfc685391/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:43:21.144214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb32cbf-681b-4545-9d35-95975f260eee None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.144503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb32cbf-681b-4545-9d35-95975f260eee None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.144816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb32cbf-681b-4545-9d35-95975f260eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.171394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7fb32cbf-681b-4545-9d35-95975f260eee None None] MFA Rules not processed for user `5147704b537744d5a7b9d80cfc685391`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:43:21.172629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7fb32cbf-681b-4545-9d35-95975f260eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:43:21.238884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 753/1502] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:43:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:43:21.252036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244212-df50-40c4-8c23-eafd706d527c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.252333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244212-df50-40c4-8c23-eafd706d527c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.252645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244212-df50-40c4-8c23-eafd706d527c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.254026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 750/1503] 10.222.0.57 () {54 vars in 963 bytes} [Fri Jul 15 17:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:43:21.258952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919719e6-1117-4104-9325-11658c047e52 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.259278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919719e6-1117-4104-9325-11658c047e52 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.259561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919719e6-1117-4104-9325-11658c047e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.317037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-919719e6-1117-4104-9325-11658c047e52 None None] MFA Rules not processed for user `4c1a2fa417e1456ba4c36d6e4dc4014d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:43:21.318254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-919719e6-1117-4104-9325-11658c047e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:43:21.393428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 754/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Fri Jul 15 17:43:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:43:21.401019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02244212-df50-40c4-8c23-eafd706d527c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:21.411425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02244212-df50-40c4-8c23-eafd706d527c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:21.415172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:21.415814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.416105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.416388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.424829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:21.426261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:21.435016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 751/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:43:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:43:21.478229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13cedc-f09f-49e3-9108-1cd916efee9e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.478527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13cedc-f09f-49e3-9108-1cd916efee9e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.478809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13cedc-f09f-49e3-9108-1cd916efee9e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.481934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 755/1506] 10.222.0.57 () {52 vars in 829 bytes} [Fri Jul 15 17:43:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:43:21.485987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:21.546008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dedf4-2aca-471f-9d26-b9a258d33db0 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:21.550628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:21.551511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.552098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.552098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] PATH_INFO: `/v3/users/5147704b537744d5a7b9d80cfc685391` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.553269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5147704b537744d5a7b9d80cfc685391)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:21.554286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:21.554286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 15 17:43:21.554286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: Jul 15 17:43:21.555507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:21.730824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5147704b537744d5a7b9d80cfc685391'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:21.743104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5147704b537744d5a7b9d80cfc685391'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:21.816653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5147704b537744d5a7b9d80cfc685391'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:21.817310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5147704b537744d5a7b9d80cfc685391'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:21.818069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5147704b537744d5a7b9d80cfc685391'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:21.834374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:43:21.842605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 752/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:43:21 2022] DELETE /identity/v3/users/5147704b537744d5a7b9d80cfc685391 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:43:21.847016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:21.847306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:21.847582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:21.866697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:43:21.867899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:43:21.877924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:43:21.927108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 756/1508] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:43:21.936929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:21.938333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:43:22.015956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91d0f3-d57c-4325-8190-2152fda04e57 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:22.019977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235e205-35b7-4ad3-886f-81d8ca47d445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:22.020655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235e205-35b7-4ad3-886f-81d8ca47d445 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:22.020948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235e205-35b7-4ad3-886f-81d8ca47d445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:22.021229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235e205-35b7-4ad3-886f-81d8ca47d445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:22.030467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235e205-35b7-4ad3-886f-81d8ca47d445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:22.031918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235e205-35b7-4ad3-886f-81d8ca47d445 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:22.041880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 753/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:43:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:43:22.561127 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:43:22.562591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:43:22.582130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468bb16-8210-4a4a-947f-04aa1e7b157b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:43:22.582130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:43:22.582130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:43:22.582130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:43:22.582471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] PATH_INFO: `/v3/projects/bac6179235774547a0b9b78af759d72d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:43:22.582471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac6179235774547a0b9b78af759d72d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:43:22.583042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:43:22.583042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:43:22.583042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:43:22.584227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:43:22.702464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac6179235774547a0b9b78af759d72d', 'request_id': 'req-c4465d64-23f1-4cb0-8f41-65ee40b139e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:22.715583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac6179235774547a0b9b78af759d72d', 'request_id': 'req-c4465d64-23f1-4cb0-8f41-65ee40b139e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:22.721674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac6179235774547a0b9b78af759d72d', 'request_id': 'req-c4465d64-23f1-4cb0-8f41-65ee40b139e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:22.814091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] The token cache is being invalidate because project bac6179235774547a0b9b78af759d72d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:43:22.814291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:22.815255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c4465d64-23f1-4cb0-8f41-65ee40b139e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:43:22.816855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 757/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:43:22 2022] DELETE /identity/v3/projects/bac6179235774547a0b9b78af759d72d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:43:32.037640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:43:32.810750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:45:02.946990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaca2d-f41b-487c-be8e-13126ce8726c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:02.947596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaca2d-f41b-487c-be8e-13126ce8726c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:02.947596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaca2d-f41b-487c-be8e-13126ce8726c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:02.949019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:02.953977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737fee9-b7f2-4562-a8cd-2f963e8b3eff None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:02.954414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737fee9-b7f2-4562-a8cd-2f963e8b3eff None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:02.954521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737fee9-b7f2-4562-a8cd-2f963e8b3eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:02.972621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4737fee9-b7f2-4562-a8cd-2f963e8b3eff None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:02.973832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4737fee9-b7f2-4562-a8cd-2f963e8b3eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:03.033216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:03.185343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaca2d-f41b-487c-be8e-13126ce8726c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:03.187149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-96aaca2d-f41b-487c-be8e-13126ce8726c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:03.291301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaca2d-f41b-487c-be8e-13126ce8726c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:03.296770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed976dec-85fa-490c-8f99-c4b434485a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:03.297417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed976dec-85fa-490c-8f99-c4b434485a93 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:03.297700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed976dec-85fa-490c-8f99-c4b434485a93 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:03.297975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed976dec-85fa-490c-8f99-c4b434485a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:03.306986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed976dec-85fa-490c-8f99-c4b434485a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:03.308628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed976dec-85fa-490c-8f99-c4b434485a93 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:03.318372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 755/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:04.171905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad68183-a498-4726-b27a-e4389d5fba4b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:04.172452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad68183-a498-4726-b27a-e4389d5fba4b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:04.172452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad68183-a498-4726-b27a-e4389d5fba4b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:04.173933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:04.180347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d1407-7905-4faf-9017-e18aba43c9eb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:04.180915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d1407-7905-4faf-9017-e18aba43c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:04.182367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d1407-7905-4faf-9017-e18aba43c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:04.201659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c36d1407-7905-4faf-9017-e18aba43c9eb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:04.202913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c36d1407-7905-4faf-9017-e18aba43c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:04.211453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:04.374555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad68183-a498-4726-b27a-e4389d5fba4b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:04.389130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad68183-a498-4726-b27a-e4389d5fba4b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:04.394652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025ab00-71d1-41c4-a0e8-4748694a988d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:04.395293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025ab00-71d1-41c4-a0e8-4748694a988d service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:04.395578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025ab00-71d1-41c4-a0e8-4748694a988d service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:04.395855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025ab00-71d1-41c4-a0e8-4748694a988d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:04.404570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025ab00-71d1-41c4-a0e8-4748694a988d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:04.406197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025ab00-71d1-41c4-a0e8-4748694a988d service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:04.416173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 760/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:05.056100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3021b-7b62-412c-8742-8b77602ac3a6 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:05.056398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3021b-7b62-412c-8742-8b77602ac3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:05.056712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3021b-7b62-412c-8742-8b77602ac3a6 None None] PATH_INFO: `/v3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:05.058053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 757/1517] 10.222.0.59 () {54 vars in 925 bytes} [Fri Jul 15 17:45:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:45:05.077933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc24650-469a-4123-96bc-951cac02c309 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:05.078227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc24650-469a-4123-96bc-951cac02c309 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:05.078510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc24650-469a-4123-96bc-951cac02c309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:05.125826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7cc24650-469a-4123-96bc-951cac02c309 None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:05.127044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7cc24650-469a-4123-96bc-951cac02c309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:05.200316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 761/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Jul 15 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:09.346023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f53b17-15fa-46fd-a654-ceb732d8cf2c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:09.346559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f53b17-15fa-46fd-a654-ceb732d8cf2c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:09.346559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f53b17-15fa-46fd-a654-ceb732d8cf2c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:09.347900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 758/1519] 10.222.0.57 () {54 vars in 960 bytes} [Fri Jul 15 17:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:09.352846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b50e-759d-4733-8b9a-f99ccf88f1dd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:09.353265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b50e-759d-4733-8b9a-f99ccf88f1dd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:09.353412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b50e-759d-4733-8b9a-f99ccf88f1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:09.372758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-41c3b50e-759d-4733-8b9a-f99ccf88f1dd None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:09.373954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-41c3b50e-759d-4733-8b9a-f99ccf88f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:09.383109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 762/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Fri Jul 15 17:45:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:09.390609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f53b17-15fa-46fd-a654-ceb732d8cf2c None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:09.404548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f53b17-15fa-46fd-a654-ceb732d8cf2c None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:09.409445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cafb3-fe35-4a42-9b80-6617f09114c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:09.410073 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cafb3-fe35-4a42-9b80-6617f09114c2 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:09.410358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cafb3-fe35-4a42-9b80-6617f09114c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:09.410635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cafb3-fe35-4a42-9b80-6617f09114c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:09.419351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cafb3-fe35-4a42-9b80-6617f09114c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:09.421014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cafb3-fe35-4a42-9b80-6617f09114c2 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:09.431185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 759/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:15.673246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd18b9-a943-472a-b3a5-465806ca5c7b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:15.673801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd18b9-a943-472a-b3a5-465806ca5c7b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:15.673801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd18b9-a943-472a-b3a5-465806ca5c7b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:15.675233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:15.680108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57c301-e6e8-4e66-8990-ba6048ea7f2b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:15.680562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57c301-e6e8-4e66-8990-ba6048ea7f2b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:15.680760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57c301-e6e8-4e66-8990-ba6048ea7f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:15.698546 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4e57c301-e6e8-4e66-8990-ba6048ea7f2b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:15.700446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4e57c301-e6e8-4e66-8990-ba6048ea7f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:15.708203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:15.862410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd18b9-a943-472a-b3a5-465806ca5c7b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:15.876913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd18b9-a943-472a-b3a5-465806ca5c7b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:15.882508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a754f53-7d1c-418d-9ac7-cd535e9ccfdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:15.883158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a754f53-7d1c-418d-9ac7-cd535e9ccfdb service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:15.883708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a754f53-7d1c-418d-9ac7-cd535e9ccfdb service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:15.883987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a754f53-7d1c-418d-9ac7-cd535e9ccfdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:15.892933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a754f53-7d1c-418d-9ac7-cd535e9ccfdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:15.895276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a754f53-7d1c-418d-9ac7-cd535e9ccfdb service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:15.905525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 764/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:16.748225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103af8-6692-4a92-8436-35304d860695 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:16.748854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103af8-6692-4a92-8436-35304d860695 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:16.748854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103af8-6692-4a92-8436-35304d860695 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:16.750210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 17:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:16.755088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603065-8624-405d-9c36-b7583f775894 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:16.755512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603065-8624-405d-9c36-b7583f775894 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:16.755643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a603065-8624-405d-9c36-b7583f775894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:16.773873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1a603065-8624-405d-9c36-b7583f775894 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:16.775034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1a603065-8624-405d-9c36-b7583f775894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:16.783015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 17:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:16.934357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103af8-6692-4a92-8436-35304d860695 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:16.948486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103af8-6692-4a92-8436-35304d860695 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:16.953980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef09c2-5740-4d66-9841-d9c40a6af8f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:16.954616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef09c2-5740-4d66-9841-d9c40a6af8f8 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:16.954902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef09c2-5740-4d66-9841-d9c40a6af8f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:16.955184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef09c2-5740-4d66-9841-d9c40a6af8f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:16.963199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef09c2-5740-4d66-9841-d9c40a6af8f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:16.964826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef09c2-5740-4d66-9841-d9c40a6af8f8 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:16.974540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 762/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:20.871895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc2aa3-6dc6-4a79-b5ef-21936f5c6fdc None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:20.873000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc2aa3-6dc6-4a79-b5ef-21936f5c6fdc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:20.873000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc2aa3-6dc6-4a79-b5ef-21936f5c6fdc None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:20.874374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:20.879645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238408-655a-43b5-a814-2a882edf3d58 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:20.880046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238408-655a-43b5-a814-2a882edf3d58 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:20.880213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238408-655a-43b5-a814-2a882edf3d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:20.898735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-06238408-655a-43b5-a814-2a882edf3d58 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:20.899934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-06238408-655a-43b5-a814-2a882edf3d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:20.907796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:20.945999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc2aa3-6dc6-4a79-b5ef-21936f5c6fdc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:20.947760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-58fc2aa3-6dc6-4a79-b5ef-21936f5c6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:20.975383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:45:21.005384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc2aa3-6dc6-4a79-b5ef-21936f5c6fdc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:21.009421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f2b9f-1ad6-4fd2-9804-87127ad4129c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:21.010277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f2b9f-1ad6-4fd2-9804-87127ad4129c service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:21.010850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f2b9f-1ad6-4fd2-9804-87127ad4129c service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:21.011139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f2b9f-1ad6-4fd2-9804-87127ad4129c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:21.020276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f2b9f-1ad6-4fd2-9804-87127ad4129c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:21.021748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f2b9f-1ad6-4fd2-9804-87127ad4129c service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:21.031221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 767/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:23.054934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb886a9c-5117-4948-90d6-9c70c359c4e7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:23.055791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb886a9c-5117-4948-90d6-9c70c359c4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:23.055791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb886a9c-5117-4948-90d6-9c70c359c4e7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:23.056903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:23.062284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde798e1-8a46-491a-a52c-2daccc0c3f6f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:23.062748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde798e1-8a46-491a-a52c-2daccc0c3f6f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:23.062855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde798e1-8a46-491a-a52c-2daccc0c3f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:23.080455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fde798e1-8a46-491a-a52c-2daccc0c3f6f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:23.084248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fde798e1-8a46-491a-a52c-2daccc0c3f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:23.089728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:23.126338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb886a9c-5117-4948-90d6-9c70c359c4e7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:23.137340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb886a9c-5117-4948-90d6-9c70c359c4e7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:23.140837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003e85c-2d1f-40bc-9114-65b9a374cbd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:23.141468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003e85c-2d1f-40bc-9114-65b9a374cbd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:23.141749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003e85c-2d1f-40bc-9114-65b9a374cbd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:23.142025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003e85c-2d1f-40bc-9114-65b9a374cbd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:23.150882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003e85c-2d1f-40bc-9114-65b9a374cbd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:23.157713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003e85c-2d1f-40bc-9114-65b9a374cbd2 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:23.162132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 765/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:26.665623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d42e9-2a82-4d98-b608-a8585cb1bcf5 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:26.666203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d42e9-2a82-4d98-b608-a8585cb1bcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:26.666203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d42e9-2a82-4d98-b608-a8585cb1bcf5 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:26.667638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:26.672791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488c5c9-6e2e-49c5-a5b8-6f63535bec52 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:26.673190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488c5c9-6e2e-49c5-a5b8-6f63535bec52 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:26.673349 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488c5c9-6e2e-49c5-a5b8-6f63535bec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:26.691502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9488c5c9-6e2e-49c5-a5b8-6f63535bec52 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:26.692736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9488c5c9-6e2e-49c5-a5b8-6f63535bec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:26.701095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:27.005160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d42e9-2a82-4d98-b608-a8585cb1bcf5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:27.006867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d66d42e9-2a82-4d98-b608-a8585cb1bcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:27.063940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d42e9-2a82-4d98-b608-a8585cb1bcf5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:27.067651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f9eb4-7a76-45c2-a771-fc5399bb10a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:27.068356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9eb4-7a76-45c2-a771-fc5399bb10a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:27.068675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9eb4-7a76-45c2-a771-fc5399bb10a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:27.068959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9eb4-7a76-45c2-a771-fc5399bb10a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:27.078292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f9eb4-7a76-45c2-a771-fc5399bb10a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:27.079732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f9eb4-7a76-45c2-a771-fc5399bb10a4 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:27.089253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 770/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:29.116635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0adeb-e3b5-47a3-a166-b579b7cff46a None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:29.117223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0adeb-e3b5-47a3-a166-b579b7cff46a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:29.117223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0adeb-e3b5-47a3-a166-b579b7cff46a None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:29.118598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 17:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:45:29.123872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0083ff7-e8b5-46fe-87a1-284c2a0ad4e6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:29.124268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0083ff7-e8b5-46fe-87a1-284c2a0ad4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:29.124439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0083ff7-e8b5-46fe-87a1-284c2a0ad4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:29.142832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d0083ff7-e8b5-46fe-87a1-284c2a0ad4e6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:45:29.144028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d0083ff7-e8b5-46fe-87a1-284c2a0ad4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:29.151838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 17:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:45:29.454416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0adeb-e3b5-47a3-a166-b579b7cff46a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:45:29.456072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-35b0adeb-e3b5-47a3-a166-b579b7cff46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:45:29.470281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0adeb-e3b5-47a3-a166-b579b7cff46a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:45:29.473934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4927e45-6856-4c78-aaf1-13ada003e159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:45:29.474576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927e45-6856-4c78-aaf1-13ada003e159 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:45:29.474862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927e45-6856-4c78-aaf1-13ada003e159 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:45:29.479788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4927e45-6856-4c78-aaf1-13ada003e159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:45:29.485028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4927e45-6856-4c78-aaf1-13ada003e159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:45:29.486463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4927e45-6856-4c78-aaf1-13ada003e159 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:45:29.496520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 768/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:45:39.138032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=47 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:45:39.491892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=82 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:00.021994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ef011-c531-4c42-adcb-0e361050a0d2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.022693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ef011-c531-4c42-adcb-0e361050a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.022693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ef011-c531-4c42-adcb-0e361050a0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.051953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b45ef011-c531-4c42-adcb-0e361050a0d2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.054555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b45ef011-c531-4c42-adcb-0e361050a0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.070107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 772/1540] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.080044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d00923-b041-4cd9-9889-08455dfa1b46 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.080658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d00923-b041-4cd9-9889-08455dfa1b46 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.080658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d00923-b041-4cd9-9889-08455dfa1b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.114168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-65d00923-b041-4cd9-9889-08455dfa1b46 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.115386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-65d00923-b041-4cd9-9889-08455dfa1b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.128556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 769/1541] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.133658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ef011-c531-4c42-adcb-0e361050a0d2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.148637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ef011-c531-4c42-adcb-0e361050a0d2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.161782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe398509-38b4-4a33-a54c-1b855e5fee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.162838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398509-38b4-4a33-a54c-1b855e5fee8c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.163360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398509-38b4-4a33-a54c-1b855e5fee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.163874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398509-38b4-4a33-a54c-1b855e5fee8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.164866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe398509-38b4-4a33-a54c-1b855e5fee8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:00.165435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe398509-38b4-4a33-a54c-1b855e5fee8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.167561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe398509-38b4-4a33-a54c-1b855e5fee8c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.186353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 773/1542] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:00.195410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d00923-b041-4cd9-9889-08455dfa1b46 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.206164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07b291-6f27-4d24-bc0e-4f2bfff4ff38 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.206939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07b291-6f27-4d24-bc0e-4f2bfff4ff38 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.207459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07b291-6f27-4d24-bc0e-4f2bfff4ff38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.209476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d00923-b041-4cd9-9889-08455dfa1b46 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.223884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.225042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.225559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.226058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.226987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.229298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.265982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6e07b291-6f27-4d24-bc0e-4f2bfff4ff38 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.267235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6e07b291-6f27-4d24-bc0e-4f2bfff4ff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.291303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 774/1543] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:00.300440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989f96-8ddb-40aa-801b-39602c1d69d9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.300777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989f96-8ddb-40aa-801b-39602c1d69d9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.301044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989f96-8ddb-40aa-801b-39602c1d69d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.330251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9b989f96-8ddb-40aa-801b-39602c1d69d9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.331437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9b989f96-8ddb-40aa-801b-39602c1d69d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.343296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 775/1544] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:00.343707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 770/1545] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:00.344959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d963b-ea8a-4d50-85bd-9f1919346307 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.349485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d963b-ea8a-4d50-85bd-9f1919346307 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.349485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d963b-ea8a-4d50-85bd-9f1919346307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.351149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.386360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-616d963b-ea8a-4d50-85bd-9f1919346307 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.387590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-616d963b-ea8a-4d50-85bd-9f1919346307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.485583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86c13-a68b-4cbd-935d-d1d8904f962e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.499029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a8c6f-4f06-4d8b-811c-3b696da3f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.499719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a8c6f-4f06-4d8b-811c-3b696da3f163 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.500005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a8c6f-4f06-4d8b-811c-3b696da3f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.500284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a8c6f-4f06-4d8b-811c-3b696da3f163 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.501903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a8c6f-4f06-4d8b-811c-3b696da3f163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.503719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-9d2a8c6f-4f06-4d8b-811c-3b696da3f163 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:47:00.503719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 15 17:47:00.503719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: Jul 15 17:47:00.505115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a8c6f-4f06-4d8b-811c-3b696da3f163 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.536478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 776/1546] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.538474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d963b-ea8a-4d50-85bd-9f1919346307 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.555123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d963b-ea8a-4d50-85bd-9f1919346307 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.561237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33281a27-4e86-447e-aeb6-8d7d38d3942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.562023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281a27-4e86-447e-aeb6-8d7d38d3942a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.562485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281a27-4e86-447e-aeb6-8d7d38d3942a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.573062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281a27-4e86-447e-aeb6-8d7d38d3942a admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.573062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33281a27-4e86-447e-aeb6-8d7d38d3942a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:00.573062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33281a27-4e86-447e-aeb6-8d7d38d3942a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.573062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33281a27-4e86-447e-aeb6-8d7d38d3942a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.579355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 777/1547] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:00.580836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd68e6-71ef-44a6-b0a1-ecc6e38e925a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.581155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd68e6-71ef-44a6-b0a1-ecc6e38e925a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.581441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd68e6-71ef-44a6-b0a1-ecc6e38e925a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.599817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f2cd68e6-71ef-44a6-b0a1-ecc6e38e925a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.601064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f2cd68e6-71ef-44a6-b0a1-ecc6e38e925a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.602598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 771/1548] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:00.604281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42810a-3364-4133-9ccf-075ffc6c97d6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.604581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42810a-3364-4133-9ccf-075ffc6c97d6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.608043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42810a-3364-4133-9ccf-075ffc6c97d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.609243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 778/1549] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.610474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18be58-c347-422a-a91f-de26268a35e9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.610770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18be58-c347-422a-a91f-de26268a35e9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.611047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18be58-c347-422a-a91f-de26268a35e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.629309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6b42810a-3364-4133-9ccf-075ffc6c97d6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.629759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0d18be58-c347-422a-a91f-de26268a35e9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.630578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6b42810a-3364-4133-9ccf-075ffc6c97d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.630934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0d18be58-c347-422a-a91f-de26268a35e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.639445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 772/1550] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.643115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 779/1551] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.643289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181aa07-486f-441b-93e3-8183a58a6dcd None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.643289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181aa07-486f-441b-93e3-8183a58a6dcd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.643289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181aa07-486f-441b-93e3-8183a58a6dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.643591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79990ba2-cd8e-4abd-9aa6-85814f5577ec None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.643591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79990ba2-cd8e-4abd-9aa6-85814f5577ec None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.643591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79990ba2-cd8e-4abd-9aa6-85814f5577ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.661575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b181aa07-486f-441b-93e3-8183a58a6dcd None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.661798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-79990ba2-cd8e-4abd-9aa6-85814f5577ec None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.662812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b181aa07-486f-441b-93e3-8183a58a6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.662975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-79990ba2-cd8e-4abd-9aa6-85814f5577ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.671156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 773/1552] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.671532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 780/1553] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.672187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79990ba2-cd8e-4abd-9aa6-85814f5577ec None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.672371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b920-5f88-4f29-a0a2-b6441e278a49 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.672675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b920-5f88-4f29-a0a2-b6441e278a49 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.672960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b920-5f88-4f29-a0a2-b6441e278a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.689170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79990ba2-cd8e-4abd-9aa6-85814f5577ec None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.691267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-aaf7b920-5f88-4f29-a0a2-b6441e278a49 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.692465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aaf7b920-5f88-4f29-a0a2-b6441e278a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.693899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe1591-e6fb-4f1c-a310-4da12756a06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.694557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe1591-e6fb-4f1c-a310-4da12756a06a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.694841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe1591-e6fb-4f1c-a310-4da12756a06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.695117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe1591-e6fb-4f1c-a310-4da12756a06a admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.695804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe1591-e6fb-4f1c-a310-4da12756a06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.696872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-1cbe1591-e6fb-4f1c-a310-4da12756a06a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:47:00.696872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:47:00.696872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:47:00.698069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe1591-e6fb-4f1c-a310-4da12756a06a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.700859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 774/1554] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.701649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7b920-5f88-4f29-a0a2-b6441e278a49 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.712509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7b920-5f88-4f29-a0a2-b6441e278a49 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.717130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b2fc0-4028-44db-ab7c-b3442cd42f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.717789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2fc0-4028-44db-ab7c-b3442cd42f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.718074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2fc0-4028-44db-ab7c-b3442cd42f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.718348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b2fc0-4028-44db-ab7c-b3442cd42f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.719103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b2fc0-4028-44db-ab7c-b3442cd42f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.720772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b2fc0-4028-44db-ab7c-b3442cd42f4e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.731336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 775/1555] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:00.732581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2528aa6-2441-426e-a8fd-452125d94b46 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.732896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2528aa6-2441-426e-a8fd-452125d94b46 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.733179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2528aa6-2441-426e-a8fd-452125d94b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.750686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b2528aa6-2441-426e-a8fd-452125d94b46 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.751867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b2528aa6-2441-426e-a8fd-452125d94b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.760239 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 776/1556] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.761092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2528aa6-2441-426e-a8fd-452125d94b46 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.770004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2528aa6-2441-426e-a8fd-452125d94b46 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.774475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.775183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.775471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.775752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.776474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:00.776811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.777824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:47:00.777824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 15 17:47:00.777824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: Jul 15 17:47:00.778706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771998d-db7c-44f4-8304-a5b20006d370 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.789966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 777/1557] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:00.791667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19628a4-82b4-4210-b0b9-207e18ad63d4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.791955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19628a4-82b4-4210-b0b9-207e18ad63d4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.792236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19628a4-82b4-4210-b0b9-207e18ad63d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.809510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b19628a4-82b4-4210-b0b9-207e18ad63d4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.810619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b19628a4-82b4-4210-b0b9-207e18ad63d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.818308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 778/1558] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.819487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802a04f-3aec-4608-9fb7-fc2af0b77004 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.819779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802a04f-3aec-4608-9fb7-fc2af0b77004 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.820163 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802a04f-3aec-4608-9fb7-fc2af0b77004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.829750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 781/1559] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:00.838723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70102556-8a11-4ed5-9f49-8e67f46d3ef2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.838723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70102556-8a11-4ed5-9f49-8e67f46d3ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.838723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70102556-8a11-4ed5-9f49-8e67f46d3ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.839283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1802a04f-3aec-4608-9fb7-fc2af0b77004 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.839902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1802a04f-3aec-4608-9fb7-fc2af0b77004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.856048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-70102556-8a11-4ed5-9f49-8e67f46d3ef2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.857344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-70102556-8a11-4ed5-9f49-8e67f46d3ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.900411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 779/1560] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.901930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d09f40-9643-4625-bd4c-6c2991f4f802 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.902222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d09f40-9643-4625-bd4c-6c2991f4f802 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.902502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d09f40-9643-4625-bd4c-6c2991f4f802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.909793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:00.920532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-17d09f40-9643-4625-bd4c-6c2991f4f802 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:00.921768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-17d09f40-9643-4625-bd4c-6c2991f4f802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:00.928421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 782/1561] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.928421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70102556-8a11-4ed5-9f49-8e67f46d3ef2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.930281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 780/1562] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:00.931649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d09f40-9643-4625-bd4c-6c2991f4f802 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.937074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70102556-8a11-4ed5-9f49-8e67f46d3ef2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.940942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d09f40-9643-4625-bd4c-6c2991f4f802 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.941930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.942611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.942898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.943174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.943904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.945495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.945648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.946140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.946457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.946739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.947466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:00.947777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.949422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.955879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 783/1563] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:00.956777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.961679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:00.962835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 781/1564] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:00.963652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:00.966835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33850f79-e547-4d4a-ad05-2622e36e0c69 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.971456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.972142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.972418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.972727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.973416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.975045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab178532-56b9-4268-9559-57a5003496a0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:00.975327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.980429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:00.981115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:00.981399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:00.981676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:00.982420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:00.982747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:00.984465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:00.997580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 782/1565] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:00.998544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.008274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831115b2-1050-4c8a-acb2-c0e05d73973f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.013937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.014674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.014959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.015233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.015918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.017859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 784/1566] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/projects => generated 393 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:01.068562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.126213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 783/1567] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:00 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:01.126213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.132516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491628-59ee-4283-96cc-83f33bcf52ae admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.135313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82d9c-16f5-4295-86fc-404e0f92945d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.137544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.138216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.138579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.138781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.139517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:01.139830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.140060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.140826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.141114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.141390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.141497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.142115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:01.142422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.145072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.155215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 785/1568] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:01.156030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.158792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 784/1569] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:01.167778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.168896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2cfc5-c130-43ba-b924-5be44af7584f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.173855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.176475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.176475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.176475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.176475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:01.176475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.177723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.180294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a37b5-a1bc-4da5-b09a-28cdbc029052 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.187665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.188708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.188951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.189347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] PATH_INFO: `/v3/projects/e07727b74ad4488bb3c60bcc5406b328/users/a8a70c43fb514e1cbecf5b4577c0c7ac/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.190849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 786/1570] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:01.191744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.202255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156f425-8800-4e3d-8010-1443c50894e1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.207095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.207457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07727b74ad4488bb3c60bcc5406b328, user_id=a8a70c43fb514e1cbecf5b4577c0c7ac, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.207745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.208032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.208312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.209046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.210423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.211009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.308146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 785/1571] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:01 2022] PUT /identity/v3/projects/e07727b74ad4488bb3c60bcc5406b328/users/a8a70c43fb514e1cbecf5b4577c0c7ac/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:01.309003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.345396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 787/1572] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:01.345396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.374556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f031067-9fc6-4a92-8b42-12cb6e8fa4c5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.407431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0c375-0fbf-4539-b5df-0f1383e47be4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.412119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.412831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.413122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.413404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.414096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.416643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.431327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.432074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.432433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.432781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.435134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.435382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.517383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 788/1573] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:01.518375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.528384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cced4-fb49-4bc6-91e4-144e0ba2c093 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.533005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.533644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.533928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.534209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.534919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.536509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.546414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 789/1574] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:01.547164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.556477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a8fe5-bc83-4ab0-a8a1-27b8970baadf admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.559084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 786/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/projects => generated 383 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:01.559937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.568855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:01.569458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:01.612423 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.613031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.613320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.613600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.614303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.616211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.619801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74d0ce-bd4f-4591-897e-1828795ace42 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.624429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd0f06-de13-42c9-b67e-65b6a723b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.625119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd0f06-de13-42c9-b67e-65b6a723b30f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.625418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd0f06-de13-42c9-b67e-65b6a723b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.625701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd0f06-de13-42c9-b67e-65b6a723b30f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.626389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd0f06-de13-42c9-b67e-65b6a723b30f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.628459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd0f06-de13-42c9-b67e-65b6a723b30f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.770642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 790/1576] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:01.770642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.828717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 787/1577] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/projects => generated 403 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:01.830185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816c7cd-a28a-4108-b09e-690b34ebfe44 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.830477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816c7cd-a28a-4108-b09e-690b34ebfe44 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.830757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816c7cd-a28a-4108-b09e-690b34ebfe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.832222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fb386-4670-4f15-af5a-c73bde57bd41 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.859311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9816c7cd-a28a-4108-b09e-690b34ebfe44 None None] MFA Rules not processed for user `a8a70c43fb514e1cbecf5b4577c0c7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:01.860527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9816c7cd-a28a-4108-b09e-690b34ebfe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:01.894275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.894864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.895157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.895493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.896194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.898213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:01.930320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 788/1578] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:01.931772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816c7cd-a28a-4108-b09e-690b34ebfe44 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:01.942456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816c7cd-a28a-4108-b09e-690b34ebfe44 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:01.948003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:01.948900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:01.949371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:01.949768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:01.950761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:01.954077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.007319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 791/1579] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/projects => generated 375 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:02.007319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.044755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:02.068672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c4d01-de3e-42bc-b048-ab9220233618 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.073989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.074660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.074948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.075234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.075968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.077607 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.087517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 792/1580] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.088308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.097947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c84f5-cafc-4f94-b1dc-8228d85f9009 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.102315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 789/1581] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:01 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:02.102315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.104182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.104873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.105161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.105441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.106170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.107770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.112118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ef4f5-b9c8-4ae6-9fc2-783e089a8690 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.117086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.117811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.118171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.118980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.119436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 793/1582] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.119555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.120246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.121697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.129948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959555ba-04fa-4aac-aab5-d080b7414fc9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.135687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ce79c-28dc-4ff3-9af5-58cda8cc7b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.136446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ce79c-28dc-4ff3-9af5-58cda8cc7b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.136774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ce79c-28dc-4ff3-9af5-58cda8cc7b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.137052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ce79c-28dc-4ff3-9af5-58cda8cc7b53 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.137819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ce79c-28dc-4ff3-9af5-58cda8cc7b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.139727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ce79c-28dc-4ff3-9af5-58cda8cc7b53 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.219875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 790/1583] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:02 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:02.220627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.230737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3963ec-31b8-4e5a-ba8f-6b55a1466ef3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.235372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.236024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.236307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.236584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.237292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.239213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.249449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 794/1584] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:02 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:02.250820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dbbe5-7388-429b-924c-61121120ddc8 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.251344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dbbe5-7388-429b-924c-61121120ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.251482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dbbe5-7388-429b-924c-61121120ddc8 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.252822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 795/1585] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 15 17:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:02.257944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dbbe5-7388-429b-924c-61121120ddc8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.270253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dbbe5-7388-429b-924c-61121120ddc8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.274830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.275459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.275749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.276030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.276789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.278728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.366780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 791/1586] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:02 2022] POST /identity/v3/users => generated 302 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:02.371005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 796/1587] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:02 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:02.371174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.371286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.384340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d86f8c-5846-4e53-a296-ec195d058a03 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.385550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c96d-87b2-4d95-bf04-388b3704199e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.390460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.391508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.391980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.392240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.392527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.393288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.393747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.393865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.394190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.395378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.396592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.398203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.409399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 792/1588] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.409758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 797/1589] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.413319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.413465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.420585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85620983-cba7-4dfa-a9a5-79061c032479 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.422187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43846d1-3cc9-4377-94e0-7f602fd3ffb6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.425446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.426289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.426578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.426850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.426973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] PATH_INFO: `/v3/projects/ae3edb73a0024048b8ad94624376f8a7/users/9c00f4aefb2b4d48a19214460e2260c9/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.427503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.427792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.428070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.428828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.430456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.441003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 798/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.441805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.442816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3edb73a0024048b8ad94624376f8a7, user_id=9c00f4aefb2b4d48a19214460e2260c9, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.445757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.449289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:02.451559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689d38a-373d-4c4d-85ad-c41e9df48893 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.460091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89afbef-ecce-4455-9d51-9456585a30b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.460804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afbef-ecce-4455-9d51-9456585a30b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.461095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afbef-ecce-4455-9d51-9456585a30b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.461375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89afbef-ecce-4455-9d51-9456585a30b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.462134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89afbef-ecce-4455-9d51-9456585a30b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.463737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89afbef-ecce-4455-9d51-9456585a30b5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.474163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 799/1591] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.475311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01a2b1-5491-4d3d-8345-9c92b09da5ab None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.475599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01a2b1-5491-4d3d-8345-9c92b09da5ab None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.475878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01a2b1-5491-4d3d-8345-9c92b09da5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.559667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7f01a2b1-5491-4d3d-8345-9c92b09da5ab None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:02.560120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 793/1592] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:02 2022] PUT /identity/v3/projects/ae3edb73a0024048b8ad94624376f8a7/users/9c00f4aefb2b4d48a19214460e2260c9/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:02.560921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7f01a2b1-5491-4d3d-8345-9c92b09da5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:02.561069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.623691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d012-d702-44ca-ba9e-e0d653bffaa1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.629227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.629930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.630354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.630642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.631940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.632071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 800/1593] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 15 17:47:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:02.632535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01a2b1-5491-4d3d-8345-9c92b09da5ab None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.633314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.642479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01a2b1-5491-4d3d-8345-9c92b09da5ab None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.647652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 794/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.647652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.650441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.651601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.651883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.652157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.652930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.654559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.658601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d82a4-9d4a-44e8-901f-d14a38152cc4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.663381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.664040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.664332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.664640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.668223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 801/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.668353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.668466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.668576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.675888 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae184-c34d-4c9a-9fa2-d25353fb3fb1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.681796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 795/1596] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.681796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.682037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.682037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.682037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.682037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] PATH_INFO: `/v3/projects/882c7039b37f45f8843752b457525d9b/users/8873b1c9f58a4adea93ebe21f5f3263d/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.688868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287f2cd-6314-42ed-bd27-2de93262b2af admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.694218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.695145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.695566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.695970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.697019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.697217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=882c7039b37f45f8843752b457525d9b, user_id=8873b1c9f58a4adea93ebe21f5f3263d, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.698134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:47:02.698134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:47:02.698134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:47:02.699382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.700098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.703284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:02.711483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 796/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.712426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.722989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47555ef9-7b26-487e-8bf8-2d4e85e2f803 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.727723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157bc7d-9367-4633-8b63-32095f6fd90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.728281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bc7d-9367-4633-8b63-32095f6fd90b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.728567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bc7d-9367-4633-8b63-32095f6fd90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.728883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bc7d-9367-4633-8b63-32095f6fd90b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.730231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157bc7d-9367-4633-8b63-32095f6fd90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.731819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157bc7d-9367-4633-8b63-32095f6fd90b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.742418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 797/1598] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.743870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b98dd1-59c3-4037-8300-ec6e2e2c76bf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.743870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b98dd1-59c3-4037-8300-ec6e2e2c76bf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.744197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b98dd1-59c3-4037-8300-ec6e2e2c76bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.768909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-39b98dd1-59c3-4037-8300-ec6e2e2c76bf None None] MFA Rules not processed for user `9c00f4aefb2b4d48a19214460e2260c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:02.770113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-39b98dd1-59c3-4037-8300-ec6e2e2c76bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:02.780243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 802/1599] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:02 2022] PUT /identity/v3/projects/882c7039b37f45f8843752b457525d9b/users/8873b1c9f58a4adea93ebe21f5f3263d/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:02.781295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.837822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 798/1600] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:02.839215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b98dd1-59c3-4037-8300-ec6e2e2c76bf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.851385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b98dd1-59c3-4037-8300-ec6e2e2c76bf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.852349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4133cb8-1e23-4b08-873e-1dc1fec3a5b3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.859280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.859670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.860501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.860879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.909948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:02.921407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.922606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.923141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.923282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.923571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.923858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.924586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.926217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.932628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 803/1601] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:02.933522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.939397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 799/1602] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.939397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.946842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754116c-986f-4cf3-9c72-42be16b6fb0b service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.949352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76ad1-caa8-4c73-93e7-64507452275d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.952202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.952886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.953169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.953440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.954135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.954272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.955045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.955334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.955632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] PATH_INFO: `/v3/projects/70f83741fb5241efad1b22e93cc67450/users/1c230254925a43ac9fe52dffbd0fe3ed/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:02.955791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.971404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 804/1603] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:02.971404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:02.975220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f83741fb5241efad1b22e93cc67450, user_id=1c230254925a43ac9fe52dffbd0fe3ed, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:02.978157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:02.981482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb27a5a-0e47-4348-b2f4-40b8312a989b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:02.986402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0f072-37ba-4214-960e-7df8920ddb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:02.987273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0f072-37ba-4214-960e-7df8920ddb71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:02.987556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0f072-37ba-4214-960e-7df8920ddb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:02.987832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0f072-37ba-4214-960e-7df8920ddb71 admin admin] PATH_INFO: `/v3/projects/b464e33662344d5a80bdfe413b56b5ec/users/34e7019b46d743a1b6b0d3315e69053f/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.003087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0f072-37ba-4214-960e-7df8920ddb71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b464e33662344d5a80bdfe413b56b5ec, user_id=34e7019b46d743a1b6b0d3315e69053f, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.006052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0f072-37ba-4214-960e-7df8920ddb71 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.051097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 800/1604] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:02 2022] PUT /identity/v3/projects/70f83741fb5241efad1b22e93cc67450/users/1c230254925a43ac9fe52dffbd0fe3ed/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:03.054945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.092186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 805/1605] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:02 2022] PUT /identity/v3/projects/b464e33662344d5a80bdfe413b56b5ec/users/34e7019b46d743a1b6b0d3315e69053f/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:03.100474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dbd92-83a8-4d46-9c60-31e39c33ee8b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.100474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dbd92-83a8-4d46-9c60-31e39c33ee8b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.100474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dbd92-83a8-4d46-9c60-31e39c33ee8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.118265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a60059-c9b0-44f7-a428-e762bc130b9f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.118484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-681dbd92-83a8-4d46-9c60-31e39c33ee8b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.119778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-681dbd92-83a8-4d46-9c60-31e39c33ee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.173665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:03.179269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.180197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.180486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.180800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] PATH_INFO: `/v3/projects/14a00a57ce8e46f0b0936cbe9c6ae9d8/users/caab1412244c43adac226cb253a10880/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.183222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 806/1606] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.184379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344bf5-dbad-4cb4-acf5-c515ffeed711 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.184702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344bf5-dbad-4cb4-acf5-c515ffeed711 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.184970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344bf5-dbad-4cb4-acf5-c515ffeed711 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.186405 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 807/1607] 10.222.0.57 () {54 vars in 961 bytes} [Fri Jul 15 17:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:03.186949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4344bf5-dbad-4cb4-acf5-c515ffeed711 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.200498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4344bf5-dbad-4cb4-acf5-c515ffeed711 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.202011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a00a57ce8e46f0b0936cbe9c6ae9d8, user_id=caab1412244c43adac226cb253a10880, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.204765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.205096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad00863-9f2c-47d7-9f59-74a96198b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.205980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00863-9f2c-47d7-9f59-74a96198b3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.206269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00863-9f2c-47d7-9f59-74a96198b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.206549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00863-9f2c-47d7-9f59-74a96198b3ac admin admin] PATH_INFO: `/v3/projects/90e68087061645019e1524959ef40957/users/caac0271896a4e5cb378407bff8378c6/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.221964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad00863-9f2c-47d7-9f59-74a96198b3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e68087061645019e1524959ef40957, user_id=caac0271896a4e5cb378407bff8378c6, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.224675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad00863-9f2c-47d7-9f59-74a96198b3ac admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.331274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 801/1608] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:03 2022] PUT /identity/v3/projects/14a00a57ce8e46f0b0936cbe9c6ae9d8/users/caab1412244c43adac226cb253a10880/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:03.332113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.386407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:03.399135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 808/1609] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:03 2022] PUT /identity/v3/projects/90e68087061645019e1524959ef40957/users/caac0271896a4e5cb378407bff8378c6/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:03.400536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d7ce9-8e34-4563-82a2-ef9719b63400 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.402805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d7ce9-8e34-4563-82a2-ef9719b63400 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.402805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d7ce9-8e34-4563-82a2-ef9719b63400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.406138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1001a4-0aca-4529-967b-a33145ae938a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.418701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-911d7ce9-8e34-4563-82a2-ef9719b63400 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.419904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-911d7ce9-8e34-4563-82a2-ef9719b63400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.466719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.467609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.467897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.468176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] PATH_INFO: `/v3/projects/fbcdf72827044c04842b74194eaf13c3/users/5013f4ba5cef44f7bdab6531ec0b0836/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.482076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbcdf72827044c04842b74194eaf13c3, user_id=5013f4ba5cef44f7bdab6531ec0b0836, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.484813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.485119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 809/1610] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.487369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a362a-c251-4ed1-9e6b-56e433f4deec None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.487659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a362a-c251-4ed1-9e6b-56e433f4deec None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.488141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a362a-c251-4ed1-9e6b-56e433f4deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.505828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-506a362a-c251-4ed1-9e6b-56e433f4deec None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.507312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-506a362a-c251-4ed1-9e6b-56e433f4deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.515508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 810/1611] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.516769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26556655-6227-4282-a15a-010b2ddfa173 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.517054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26556655-6227-4282-a15a-010b2ddfa173 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.517370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26556655-6227-4282-a15a-010b2ddfa173 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.520739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 811/1612] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 15 17:47:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:03.522123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4ab06-8f4e-45d9-b0e8-d0cc79732c1e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.522588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4ab06-8f4e-45d9-b0e8-d0cc79732c1e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.522876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4ab06-8f4e-45d9-b0e8-d0cc79732c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.543996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-57f4ab06-8f4e-45d9-b0e8-d0cc79732c1e None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.545255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-57f4ab06-8f4e-45d9-b0e8-d0cc79732c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.585604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 802/1613] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:03 2022] PUT /identity/v3/projects/fbcdf72827044c04842b74194eaf13c3/users/5013f4ba5cef44f7bdab6531ec0b0836/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:03.586664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.588241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.613055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 812/1614] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.615010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd20700-4979-4506-9644-afb674f493db None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.615305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd20700-4979-4506-9644-afb674f493db None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.615585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd20700-4979-4506-9644-afb674f493db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.633324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7fd20700-4979-4506-9644-afb674f493db None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.634519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7fd20700-4979-4506-9644-afb674f493db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.647130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acc1d6-5f6d-4fc6-bc0b-2bddb03e5ee8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.650482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6d3aa-74c2-42eb-a40d-8759070d5a77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.651157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6d3aa-74c2-42eb-a40d-8759070d5a77 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.651443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6d3aa-74c2-42eb-a40d-8759070d5a77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.651720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6d3aa-74c2-42eb-a40d-8759070d5a77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.687954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:03.696011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 813/1615] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.697372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a6e02-05c1-49f9-b205-f683666c414d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.697699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a6e02-05c1-49f9-b205-f683666c414d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.698045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a6e02-05c1-49f9-b205-f683666c414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.715862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-468a6e02-05c1-49f9-b205-f683666c414d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.717099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-468a6e02-05c1-49f9-b205-f683666c414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.717236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6d3aa-74c2-42eb-a40d-8759070d5a77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.718647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6d3aa-74c2-42eb-a40d-8759070d5a77 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.725674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 814/1616] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.727279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a6e02-05c1-49f9-b205-f683666c414d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.733322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 803/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:03.734872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253d8e-f01d-45ad-b79b-6420f1ff7bda None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.735165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253d8e-f01d-45ad-b79b-6420f1ff7bda None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.735442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db253d8e-f01d-45ad-b79b-6420f1ff7bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.739484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a6e02-05c1-49f9-b205-f683666c414d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.743571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.744299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.744584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.745614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.755178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.755538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-db253d8e-f01d-45ad-b79b-6420f1ff7bda None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.757026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db253d8e-f01d-45ad-b79b-6420f1ff7bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.757152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.767341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 815/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:03.768011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.776778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a3168-ba28-4093-b4fc-bc9f0d7fd88e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.780549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc264ba-d908-4df7-b55d-0a8d9037ad62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.781220 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc264ba-d908-4df7-b55d-0a8d9037ad62 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.781509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc264ba-d908-4df7-b55d-0a8d9037ad62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.781789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc264ba-d908-4df7-b55d-0a8d9037ad62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.786894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:03.793188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc264ba-d908-4df7-b55d-0a8d9037ad62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.794657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc264ba-d908-4df7-b55d-0a8d9037ad62 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.804714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 816/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:03.809471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5d3b-5452-4f41-b06a-29a7068eeab9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.809471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5d3b-5452-4f41-b06a-29a7068eeab9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.809471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc5d3b-5452-4f41-b06a-29a7068eeab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.824351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-00dc5d3b-5452-4f41-b06a-29a7068eeab9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:03.825622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-00dc5d3b-5452-4f41-b06a-29a7068eeab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:03.833874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 817/1620] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.834766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5d3b-5452-4f41-b06a-29a7068eeab9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.846494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 804/1621] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:03.847565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db253d8e-f01d-45ad-b79b-6420f1ff7bda None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.855295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc5d3b-5452-4f41-b06a-29a7068eeab9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.857541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db253d8e-f01d-45ad-b79b-6420f1ff7bda None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.861578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.861952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.862078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.862302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.862509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.862648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.862917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.863240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.875939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.878833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.888743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 805/1622] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:03.889436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.901816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ffc70b-0910-40e8-8d79-9755d2f62e9f service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.909829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.910542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.911278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.911278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.925896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.927389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.939520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 806/1623] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:03.940169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.943805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.945989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.949667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5f608-eb96-458b-b4a8-91145452f491 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:03.954342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbba175-fc97-4d33-ba44-c09af6a7a9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:03.955539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbba175-fc97-4d33-ba44-c09af6a7a9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:03.956146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbba175-fc97-4d33-ba44-c09af6a7a9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:03.957359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbba175-fc97-4d33-ba44-c09af6a7a9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:03.959073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 818/1624] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:03.960048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:03.980297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbba175-fc97-4d33-ba44-c09af6a7a9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:03.982348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbba175-fc97-4d33-ba44-c09af6a7a9b1 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:03.996659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 807/1625] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:04.042214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca4a8e-3e70-4667-94f0-acd317365dff service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:04.046084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaea5a-fd2a-4450-bbe8-0b5af402c980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:04.047131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaea5a-fd2a-4450-bbe8-0b5af402c980 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:04.047131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaea5a-fd2a-4450-bbe8-0b5af402c980 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:04.047333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaea5a-fd2a-4450-bbe8-0b5af402c980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:04.059418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaea5a-fd2a-4450-bbe8-0b5af402c980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:04.060970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaea5a-fd2a-4450-bbe8-0b5af402c980 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:04.073434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 819/1626] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:04.219569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a65e5-7bde-42d8-ae41-77e7b0d2dcf2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:04.220054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a65e5-7bde-42d8-ae41-77e7b0d2dcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:04.220439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a65e5-7bde-42d8-ae41-77e7b0d2dcf2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:04.222321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 808/1627] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 15 17:47:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:04.235318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa56587-b46c-487c-9446-0533356af1c3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:04.235318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa56587-b46c-487c-9446-0533356af1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:04.235318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa56587-b46c-487c-9446-0533356af1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:04.262128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bfa56587-b46c-487c-9446-0533356af1c3 None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:04.264454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bfa56587-b46c-487c-9446-0533356af1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:04.283212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 820/1628] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:47:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:04.292963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a65e5-7bde-42d8-ae41-77e7b0d2dcf2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:04.304767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a65e5-7bde-42d8-ae41-77e7b0d2dcf2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:04.309595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:04.310397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:04.310759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:04.311108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:04.322593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:04.324175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:04.335855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 809/1629] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:06.750195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ffc43-aa8f-4888-83b3-39a00772af9c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:06.752010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ffc43-aa8f-4888-83b3-39a00772af9c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:06.753223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ffc43-aa8f-4888-83b3-39a00772af9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:06.797829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f76ffc43-aa8f-4888-83b3-39a00772af9c None None] MFA Rules not processed for user `caac0271896a4e5cb378407bff8378c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:06.799095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f76ffc43-aa8f-4888-83b3-39a00772af9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:07.075759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:07.075759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 821/1630] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:07.076808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:07.076808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed09361-1eaa-4a06-961b-53b0c1c65aa5 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:07.076808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:07.076808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:07.077850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:07.077850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:07.077850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:07.077850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:07.209517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 810/1631] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:06 2022] POST /identity/v3/projects => generated 405 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:07.211384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ffc43-aa8f-4888-83b3-39a00772af9c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:07.227893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ffc43-aa8f-4888-83b3-39a00772af9c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:07.321091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:07.321694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:07.322596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:07.322980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:07.324284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:07.327120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:07.433474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 822/1632] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:07 2022] POST /identity/v3/users => generated 309 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:07.439074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:07.470979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69222d8f-e634-4e6c-ae77-cc94ed5910e9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:07.479174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:07.480310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:07.481947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:07.483081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:07.484240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:07.486753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:07.512245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 811/1633] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:07.516665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:07.536215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c854f-d6c0-42c1-b4f9-860adc759422 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:07.545612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:07.546858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:07.547391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:07.547817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] PATH_INFO: `/v3/projects/86e6c7dd14994aaa9d6ca68d1a0e7713/users/f68422d627da4356803d5aae6c6f158d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:07.580808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e6c7dd14994aaa9d6ca68d1a0e7713, user_id=f68422d627da4356803d5aae6c6f158d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:07.590609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:07.758911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 823/1634] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:07 2022] PUT /identity/v3/projects/86e6c7dd14994aaa9d6ca68d1a0e7713/users/f68422d627da4356803d5aae6c6f158d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:07.766359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:07.779696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54359a7-9519-4e5b-a721-22ad10f257ea admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:08.002323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:08.003014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:08.003306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:08.003585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:08.004311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:08.013868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:08.033144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 812/1635] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:08.037970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:08.057763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a30ec-5754-48bf-82bf-e28783eac78e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:08.069112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632eadb3-504c-428b-bd28-f6fb04203afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:08.070322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eadb3-504c-428b-bd28-f6fb04203afd admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:08.070870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eadb3-504c-428b-bd28-f6fb04203afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:08.071612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eadb3-504c-428b-bd28-f6fb04203afd admin admin] PATH_INFO: `/v3/projects/86e6c7dd14994aaa9d6ca68d1a0e7713/users/f68422d627da4356803d5aae6c6f158d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:08.074816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632eadb3-504c-428b-bd28-f6fb04203afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e6c7dd14994aaa9d6ca68d1a0e7713, user_id=f68422d627da4356803d5aae6c6f158d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:08.080290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632eadb3-504c-428b-bd28-f6fb04203afd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:08.120847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 824/1636] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:08 2022] PUT /identity/v3/projects/86e6c7dd14994aaa9d6ca68d1a0e7713/users/f68422d627da4356803d5aae6c6f158d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:09.396765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:09.408390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0968e72-1aaa-4ba5-90e4-d4f423cf33cf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:09.481329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:09.538333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:09.539607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:09.540088 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:09.540697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] PATH_INFO: `/v3/users/a8a70c43fb514e1cbecf5b4577c0c7ac` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:09.546417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a70c43fb514e1cbecf5b4577c0c7ac)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:09.555813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:09.858858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a70c43fb514e1cbecf5b4577c0c7ac'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:09.868303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a70c43fb514e1cbecf5b4577c0c7ac'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:10.005871 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a70c43fb514e1cbecf5b4577c0c7ac'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:10.007056 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a70c43fb514e1cbecf5b4577c0c7ac'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:10.007852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a70c43fb514e1cbecf5b4577c0c7ac'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:10.046376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 813/1637] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:09 2022] DELETE /identity/v3/users/a8a70c43fb514e1cbecf5b4577c0c7ac => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:10.052288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0644bb-50ad-468a-a487-c7f5b2ef43ad None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:10.053134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0644bb-50ad-468a-a487-c7f5b2ef43ad None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:10.053552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0644bb-50ad-468a-a487-c7f5b2ef43ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:10.089674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1a0644bb-50ad-468a-a487-c7f5b2ef43ad None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:10.096391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1a0644bb-50ad-468a-a487-c7f5b2ef43ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:10.322828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 825/1638] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:10.426304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:10.430275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:10.753437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7dfd9-35e2-41c7-9ee2-ce7da6161f4f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:10.783597 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:10.788256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:10.788546 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:10.788856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:10.814508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:10.816344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:10.840355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 814/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:12.676811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066e730-6e82-4486-85c7-89ac04bac168 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:12.677383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066e730-6e82-4486-85c7-89ac04bac168 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:12.677622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066e730-6e82-4486-85c7-89ac04bac168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:12.759451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8066e730-6e82-4486-85c7-89ac04bac168 None None] MFA Rules not processed for user `f68422d627da4356803d5aae6c6f158d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:12.764044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8066e730-6e82-4486-85c7-89ac04bac168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:12.808990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:12.837548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:12.840397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:12.875686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc2bf4-403c-436b-960d-94bc50acecd6 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:12.898334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:12.899391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:12.902242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:12.902659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] PATH_INFO: `/v3/projects/e07727b74ad4488bb3c60bcc5406b328` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:12.904925 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07727b74ad4488bb3c60bcc5406b328)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:12.907861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: WARNING py.warnings [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:47:12.907861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: warnings.warn(msg) Jul 15 17:47:12.907861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: Jul 15 17:47:12.919861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:13.017964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 826/1640] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:12 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:13.229928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07727b74ad4488bb3c60bcc5406b328', 'request_id': 'req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:13.235977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:13.250057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07727b74ad4488bb3c60bcc5406b328', 'request_id': 'req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:13.387689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07727b74ad4488bb3c60bcc5406b328', 'request_id': 'req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:13.407059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] The token cache is being invalidate because project e07727b74ad4488bb3c60bcc5406b328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:13.407721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:13.409647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9924ca4e-cd2b-458a-aa3e-de4183a03fdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:13.413933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 815/1641] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:12 2022] DELETE /identity/v3/projects/e07727b74ad4488bb3c60bcc5406b328 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:13.420022 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84316658-5891-4561-9d41-d22f16679066 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:13.421133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84316658-5891-4561-9d41-d22f16679066 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:13.421618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84316658-5891-4561-9d41-d22f16679066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:13.448286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-84316658-5891-4561-9d41-d22f16679066 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:13.449983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-84316658-5891-4561-9d41-d22f16679066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:13.527033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 827/1642] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:13.530849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e490c5f-7be0-4723-88a4-91814351507f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:13.531388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e490c5f-7be0-4723-88a4-91814351507f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:13.532069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e490c5f-7be0-4723-88a4-91814351507f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:13.609466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4e490c5f-7be0-4723-88a4-91814351507f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:13.611663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4e490c5f-7be0-4723-88a4-91814351507f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:13.635855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 816/1643] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:13.640378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84316658-5891-4561-9d41-d22f16679066 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:13.650727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84316658-5891-4561-9d41-d22f16679066 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:13.673448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:13.674362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:13.674821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:13.675279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:13.676562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:13.677084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:13.679368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:13.700405 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 828/1644] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:13.701272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e490c5f-7be0-4723-88a4-91814351507f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:13.714891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e490c5f-7be0-4723-88a4-91814351507f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:13.730666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:13.731388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:13.731687 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:13.731976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:13.732744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:13.734721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:13.951727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 817/1645] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:13 2022] POST /identity/v3/projects => generated 371 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:47:13.956908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:13.980380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5067c0-6b10-4e73-a4e2-e44f1e1ddc63 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:14.122599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:14.124332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:14.124885 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:14.125361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:14.126784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:14.129056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:14.273664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 829/1646] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:13 2022] POST /identity/v3/users => generated 286 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:14.279822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:14.297858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebc1be-e8db-4104-89e2-1792acb66e15 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:14.313809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:14.314633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:14.315095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:14.315818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:14.318919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:14.321817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:14.359803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 818/1647] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:14.364004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:14.378314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea68eb-7896-4fbc-82ee-b1f6970e6d77 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:14.409544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886c3b-6ff3-42b2-bd54-269aa08d5adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:14.410291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886c3b-6ff3-42b2-bd54-269aa08d5adf admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:14.410584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886c3b-6ff3-42b2-bd54-269aa08d5adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:14.410869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886c3b-6ff3-42b2-bd54-269aa08d5adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:14.411638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886c3b-6ff3-42b2-bd54-269aa08d5adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:14.413339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46886c3b-6ff3-42b2-bd54-269aa08d5adf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:14.429956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 830/1648] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:47:14.435681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:14.448412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70451937-4925-4a0e-b0ff-452bd50ff054 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:14.477904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:14.479013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:14.479298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:14.479575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] PATH_INFO: `/v3/projects/c5871a7531d64d3297241a0461e0d631/users/df563d10d16340fb9c4e32478fd5df8b/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:14.523745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5871a7531d64d3297241a0461e0d631, user_id=df563d10d16340fb9c4e32478fd5df8b, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:14.531462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:14.638795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 819/1649] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:14 2022] PUT /identity/v3/projects/c5871a7531d64d3297241a0461e0d631/users/df563d10d16340fb9c4e32478fd5df8b/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:14.649223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232222ad-e560-4212-beff-caee648a19c4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:14.649513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232222ad-e560-4212-beff-caee648a19c4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:14.649863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232222ad-e560-4212-beff-caee648a19c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:14.685564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-232222ad-e560-4212-beff-caee648a19c4 None None] MFA Rules not processed for user `df563d10d16340fb9c4e32478fd5df8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:14.688531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-232222ad-e560-4212-beff-caee648a19c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:14.729749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:14.877355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 831/1650] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:14.889943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:14.894268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:15.093484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e34d71-b00e-46d9-b91b-201a812bc797 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:15.122278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478e19-39a5-452e-b669-75d8cf586890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:15.122962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478e19-39a5-452e-b669-75d8cf586890 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:15.123833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478e19-39a5-452e-b669-75d8cf586890 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:15.124449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478e19-39a5-452e-b669-75d8cf586890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:15.148081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f478e19-39a5-452e-b669-75d8cf586890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:15.151276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f478e19-39a5-452e-b669-75d8cf586890 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:15.189098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 820/1651] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:15.781260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232222ad-e560-4212-beff-caee648a19c4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:15.934527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232222ad-e560-4212-beff-caee648a19c4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:15.944485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:15.945947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:15.946399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:15.946853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] PATH_INFO: `/v3/users/df563d10d16340fb9c4e32478fd5df8b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:15.949035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=df563d10d16340fb9c4e32478fd5df8b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:15.950283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: WARNING py.warnings [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 17:47:15.950283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: warnings.warn(msg) Jul 15 17:47:15.950283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: Jul 15 17:47:15.952124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:16.350278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df563d10d16340fb9c4e32478fd5df8b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:16.428555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df563d10d16340fb9c4e32478fd5df8b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:16.429486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df563d10d16340fb9c4e32478fd5df8b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:16.470542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df563d10d16340fb9c4e32478fd5df8b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:16.473975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d1132a2a-d047-43b8-a65d-b666ea1de14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df563d10d16340fb9c4e32478fd5df8b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:16.587154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 832/1652] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:15 2022] DELETE /identity/v3/users/df563d10d16340fb9c4e32478fd5df8b => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:16.592391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:16.592859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:16.593136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:16.621846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:16.623468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:16.733855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:16.759656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 821/1653] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:16.783293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1bf00-5a8b-428a-b17c-175e999b9965 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:16.783293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1bf00-5a8b-428a-b17c-175e999b9965 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:16.783293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1bf00-5a8b-428a-b17c-175e999b9965 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:16.783293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 833/1654] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:16.810804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:16.812474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:16.932287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1bf00-5a8b-428a-b17c-175e999b9965 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:16.932287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fac1bf00-5a8b-428a-b17c-175e999b9965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:16.975475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1bf00-5a8b-428a-b17c-175e999b9965 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:16.981455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:16.982867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:16.983248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:16.983695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] PATH_INFO: `/v3/users/9c00f4aefb2b4d48a19214460e2260c9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:16.985415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c00f4aefb2b4d48a19214460e2260c9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:16.987725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:16.999160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1ef1b-66de-4d14-aa5b-e5ed6c735dde None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:17.019544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4ba52-28a3-4ffb-9faf-c60671aff40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:17.020288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4ba52-28a3-4ffb-9faf-c60671aff40e service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.020587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4ba52-28a3-4ffb-9faf-c60671aff40e service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.022887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4ba52-28a3-4ffb-9faf-c60671aff40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.052278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4ba52-28a3-4ffb-9faf-c60671aff40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:17.054008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4ba52-28a3-4ffb-9faf-c60671aff40e service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:17.080743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 822/1655] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:17.082187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07dc05-07e2-4240-8083-774ac78c408e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.082711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07dc05-07e2-4240-8083-774ac78c408e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.083069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07dc05-07e2-4240-8083-774ac78c408e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.124038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ad07dc05-07e2-4240-8083-774ac78c408e None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:17.132935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ad07dc05-07e2-4240-8083-774ac78c408e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:17.194960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:17.221113 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c00f4aefb2b4d48a19214460e2260c9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:17.265755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c00f4aefb2b4d48a19214460e2260c9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:17.266824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c00f4aefb2b4d48a19214460e2260c9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:17.289242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c00f4aefb2b4d48a19214460e2260c9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:17.293014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1301c318-ac6a-46dd-a112-2a02546a7901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c00f4aefb2b4d48a19214460e2260c9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:17.327345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 823/1656] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:47:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:17.424204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911cb21-cfbd-4ed3-b77f-5517dd79473f None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.424891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911cb21-cfbd-4ed3-b77f-5517dd79473f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.425284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911cb21-cfbd-4ed3-b77f-5517dd79473f None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.427402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 824/1657] 10.222.0.57 () {54 vars in 960 bytes} [Fri Jul 15 17:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:47:17.436394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79fffb-fcaa-4185-86e0-93f36aa8308c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.436394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79fffb-fcaa-4185-86e0-93f36aa8308c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.436811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79fffb-fcaa-4185-86e0-93f36aa8308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.546246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 834/1658] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:16 2022] DELETE /identity/v3/users/9c00f4aefb2b4d48a19214460e2260c9 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:17.547493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0b79fffb-fcaa-4185-86e0-93f36aa8308c None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:17.547493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0b79fffb-fcaa-4185-86e0-93f36aa8308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:17.547652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fd8b-657d-4bad-bf8b-427818b8d03e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.547652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fd8b-657d-4bad-bf8b-427818b8d03e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.547652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fd8b-657d-4bad-bf8b-427818b8d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.547652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e257fd8b-657d-4bad-bf8b-427818b8d03e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:17.547652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e257fd8b-657d-4bad-bf8b-427818b8d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:17.706414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 825/1659] 10.222.0.57 () {58 vars in 1057 bytes} [Fri Jul 15 17:47:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:17.707934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.708406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.708740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.770995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 835/1660] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:17.773103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257fd8b-657d-4bad-bf8b-427818b8d03e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:17.778752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] MFA Rules not processed for user `34e7019b46d743a1b6b0d3315e69053f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:17.780275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:17.802282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257fd8b-657d-4bad-bf8b-427818b8d03e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:17.809887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:17.810719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.813219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.813646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.827074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:17.829387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:17.851087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 836/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:17.852097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:17.855072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:17.933253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3d9a0-da5f-4163-b6ea-857f8dd8ea0c service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:17.938898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 826/1662] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:17.944820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b5124-6b51-41c9-b749-985d524fc234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:17.944820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b5124-6b51-41c9-b749-985d524fc234 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:17.944820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b5124-6b51-41c9-b749-985d524fc234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:17.944820 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b5124-6b51-41c9-b749-985d524fc234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:17.955300 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:17.962600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:17.962722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b5124-6b51-41c9-b749-985d524fc234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:17.964134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b5124-6b51-41c9-b749-985d524fc234 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:17.985777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 837/1663] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:18.115123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90aa10-e4d9-4bf1-9515-ecbf5758c7c8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:18.129835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:18.130634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:18.131066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:18.132056 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:18.145811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:18.149017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:18.168901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 827/1664] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:18.184739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df616bf-d679-499f-ace5-1a15933ee3a6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:18.185192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df616bf-d679-499f-ace5-1a15933ee3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:18.185774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df616bf-d679-499f-ace5-1a15933ee3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:18.250518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0df616bf-d679-499f-ace5-1a15933ee3a6 None None] MFA Rules not processed for user `5013f4ba5cef44f7bdab6531ec0b0836`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:18.251756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0df616bf-d679-499f-ace5-1a15933ee3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:18.387802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 838/1665] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:18.406442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:18.409057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:18.435280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538833-1bb3-4afa-bf54-4b326f12ea67 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:18.446365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd27b0b-5391-4034-836e-bd052cb2e981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:18.447852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27b0b-5391-4034-836e-bd052cb2e981 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:18.448476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27b0b-5391-4034-836e-bd052cb2e981 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:18.448876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd27b0b-5391-4034-836e-bd052cb2e981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:18.459940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd27b0b-5391-4034-836e-bd052cb2e981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:18.462055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd27b0b-5391-4034-836e-bd052cb2e981 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:18.485697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 828/1666] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:47:18.586551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:19.182506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:19.259625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9786e55-961d-41cc-9539-06293cedd67b None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:19.260272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9786e55-961d-41cc-9539-06293cedd67b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:19.260741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9786e55-961d-41cc-9539-06293cedd67b None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:19.262621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 839/1667] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:19.293742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:19.294285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:19.294576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:19.334604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] MFA Rules not processed for user `b79bed7026ac4124a1228bc06f298c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:19.334604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:19.357436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 829/1668] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:19.401745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9786e55-961d-41cc-9539-06293cedd67b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:19.428135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9786e55-961d-41cc-9539-06293cedd67b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:19.453221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:19.454326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:19.455964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:19.456443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:19.476394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:19.482753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:19.519360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 840/1669] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:19.602191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:19.603502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:19.633760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78149761-35d7-42bf-967f-c852dcee9e33 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:19.646844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:19.649971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:19.671412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:19.671412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:19.671412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:19.671412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] PATH_INFO: `/v3/users/caac0271896a4e5cb378407bff8378c6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:19.674302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] RBAC: Authorizing `identity:delete_user(user_id=caac0271896a4e5cb378407bff8378c6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:19.682985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:19.687208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63596200-a54a-4842-afea-ad70480bb71c service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:19.703423 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:19.705900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:19.706305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:19.706697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] PATH_INFO: `/v3/projects/c5871a7531d64d3297241a0461e0d631` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:19.709067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5871a7531d64d3297241a0461e0d631)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:19.714647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:20.036663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5871a7531d64d3297241a0461e0d631', 'request_id': 'req-8bdff437-ba70-40a2-b673-1615cfa190f7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.036830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caac0271896a4e5cb378407bff8378c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.067143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5871a7531d64d3297241a0461e0d631', 'request_id': 'req-8bdff437-ba70-40a2-b673-1615cfa190f7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.067367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caac0271896a4e5cb378407bff8378c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.092988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5871a7531d64d3297241a0461e0d631', 'request_id': 'req-8bdff437-ba70-40a2-b673-1615cfa190f7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.185286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caac0271896a4e5cb378407bff8378c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.186528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caac0271896a4e5cb378407bff8378c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.187934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caac0271896a4e5cb378407bff8378c6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.222800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 830/1670] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:19 2022] DELETE /identity/v3/users/caac0271896a4e5cb378407bff8378c6 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:20.223676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:20.229788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:20.241920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:20.298298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] The token cache is being invalidate because project c5871a7531d64d3297241a0461e0d631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:20.298860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.300294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8bdff437-ba70-40a2-b673-1615cfa190f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.303640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 841/1671] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:19 2022] DELETE /identity/v3/projects/c5871a7531d64d3297241a0461e0d631 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:47:20.309292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60cc7a-8e6b-4fc8-991d-bf0b9ef478e6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:20.310181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60cc7a-8e6b-4fc8-991d-bf0b9ef478e6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:20.312139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60cc7a-8e6b-4fc8-991d-bf0b9ef478e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:20.369887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cb60cc7a-8e6b-4fc8-991d-bf0b9ef478e6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:20.372373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cb60cc7a-8e6b-4fc8-991d-bf0b9ef478e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:20.400372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 842/1672] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:20.402350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4eec76-7c1b-41a2-8570-7c0247d6b1f0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:20.404944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3ea5f-36e2-494c-a948-324c7799900b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:20.405862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4eec76-7c1b-41a2-8570-7c0247d6b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:20.406360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4eec76-7c1b-41a2-8570-7c0247d6b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:20.422425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:20.423651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:20.424157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:20.424809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] PATH_INFO: `/v3/projects/ae3edb73a0024048b8ad94624376f8a7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:20.428207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3edb73a0024048b8ad94624376f8a7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:20.437972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:20.451943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7e4eec76-7c1b-41a2-8570-7c0247d6b1f0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:20.453501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7e4eec76-7c1b-41a2-8570-7c0247d6b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:20.475551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 843/1673] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:20.481585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4eec76-7c1b-41a2-8570-7c0247d6b1f0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:20.515137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4eec76-7c1b-41a2-8570-7c0247d6b1f0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:20.522228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:20.526475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:20.526475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:20.526475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:20.526475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:20.526475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:20.528763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:20.559394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 844/1674] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:20.563513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:20.582593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a967d4-b50c-426f-ad0a-3c178aa257ff admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:20.593880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:20.594920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:20.595379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:20.595851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:20.596943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:20.599119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:20.718930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3edb73a0024048b8ad94624376f8a7', 'request_id': 'req-4e9bc009-b597-47ea-8a01-e99cf027503d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.735689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3edb73a0024048b8ad94624376f8a7', 'request_id': 'req-4e9bc009-b597-47ea-8a01-e99cf027503d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.766606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 845/1675] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:20 2022] POST /identity/v3/projects => generated 403 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:20.770812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:20.827246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3edb73a0024048b8ad94624376f8a7', 'request_id': 'req-4e9bc009-b597-47ea-8a01-e99cf027503d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.844530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] The token cache is being invalidate because project ae3edb73a0024048b8ad94624376f8a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:20.844828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.849014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e9bc009-b597-47ea-8a01-e99cf027503d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:20.851617 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 831/1676] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:20 2022] DELETE /identity/v3/projects/ae3edb73a0024048b8ad94624376f8a7 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:20.857320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63c95f-a04c-408c-8cf8-8c29985f9588 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:20.857610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63c95f-a04c-408c-8cf8-8c29985f9588 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:20.860264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63c95f-a04c-408c-8cf8-8c29985f9588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:20.929200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ee63c95f-a04c-408c-8cf8-8c29985f9588 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:20.929200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ee63c95f-a04c-408c-8cf8-8c29985f9588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:20.962144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2d4b3-43b6-461e-ab57-0116c7080f97 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:20.986505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:20.987215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:20.987509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:20.987782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:20.988846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:20.990534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.043279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:21.141391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 832/1677] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:21.144476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81107-b970-4bba-b63b-e3bc058c1fab None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.144476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81107-b970-4bba-b63b-e3bc058c1fab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.144869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81107-b970-4bba-b63b-e3bc058c1fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.175534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 846/1678] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:20 2022] POST /identity/v3/users => generated 302 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:21.179268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.183273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:21.186061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3ad81107-b970-4bba-b63b-e3bc058c1fab None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:21.187541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3ad81107-b970-4bba-b63b-e3bc058c1fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:21.212784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 833/1679] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:21.213994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81107-b970-4bba-b63b-e3bc058c1fab None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.218878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9388b9-071d-4686-8083-44bf0b702cb7 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.231281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81107-b970-4bba-b63b-e3bc058c1fab None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.252700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.255547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.256031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.256817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.258035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.258988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.259107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:21.259285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.259471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.259584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.260899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.262852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.262945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.285768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 834/1680] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:47:21.287144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.309939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 847/1681] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:21.311142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4161e-7c99-4c1b-9d6f-7c51843ff5ee admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.312851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.318271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.319229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.319629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.319922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.323124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.325299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.341447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24cda0-72f4-4b04-9691-07dc433ccd85 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.373524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.373524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.373524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.373524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.373845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.377471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.403378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 848/1682] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:21.404377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.418051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aefff-fd02-4912-942c-433bd2d1dfa0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.431557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d883522-aa9d-41dd-b29c-6c82c05f7cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.433771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d883522-aa9d-41dd-b29c-6c82c05f7cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.436213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d883522-aa9d-41dd-b29c-6c82c05f7cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.436989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d883522-aa9d-41dd-b29c-6c82c05f7cbe admin admin] PATH_INFO: `/v3/projects/2a6f8eb5152e4991bcc2d893b5c062be/users/2c65e34f5aad4f468e1706a5790bb274/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.474317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 835/1683] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:21 2022] POST /identity/v3/projects => generated 355 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:21.474968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d883522-aa9d-41dd-b29c-6c82c05f7cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6f8eb5152e4991bcc2d893b5c062be, user_id=2c65e34f5aad4f468e1706a5790bb274, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.478203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.482487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d883522-aa9d-41dd-b29c-6c82c05f7cbe admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.618017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8447b-0b4d-47b2-8e16-432db1141513 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.633990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 849/1684] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:21 2022] PUT /identity/v3/projects/2a6f8eb5152e4991bcc2d893b5c062be/users/2c65e34f5aad4f468e1706a5790bb274/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:21.644909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f9ca-4589-4344-9b4f-ef5767758efa None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.646146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f9ca-4589-4344-9b4f-ef5767758efa None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.646632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f9ca-4589-4344-9b4f-ef5767758efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.729535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-61c9f9ca-4589-4344-9b4f-ef5767758efa None None] MFA Rules not processed for user `2c65e34f5aad4f468e1706a5790bb274`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:21.730792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-61c9f9ca-4589-4344-9b4f-ef5767758efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:21.768194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.768978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.769262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.769702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.770568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.774549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.917518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 850/1685] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:21.919777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f9ca-4589-4344-9b4f-ef5767758efa None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.924422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 836/1686] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:21 2022] POST /identity/v3/users => generated 278 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:21.930775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:21.953347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f9ca-4589-4344-9b4f-ef5767758efa None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.956765 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85c2ae-d07c-4af9-a456-a58c7e1dd572 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:21.969031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.969715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.970003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.970550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.974246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.974246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:21.982688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:21.983386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:21.983676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:21.983956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:21.985371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:21.986650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:22.017890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 837/1687] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:22.021948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:22.032476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:22.047672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1d000-32ed-4569-acd7-5fcef3258d78 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:22.062153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:22.062799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:22.064822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:22.065146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:22.065935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:22.077901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:22.105960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 851/1688] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:21 2022] POST /identity/v3/projects => generated 405 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:22.107524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:22.112368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:22.121762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 838/1689] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:22.121762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:22.156781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5965c2-3e82-42d1-a90f-a11929b5f47a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:22.367169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ffef2-af47-4843-869b-29785a911b2e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:22.367926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:22.369412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:22.370028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:22.371876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:22.372892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:22.375190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:22.389626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:22.394041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:22.394041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:22.394041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] PATH_INFO: `/v3/projects/90e68087061645019e1524959ef40957` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:22.394041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e68087061645019e1524959ef40957)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:22.396188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:22.491330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 839/1690] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:22 2022] POST /identity/v3/users => generated 309 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:22.493732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:22.525791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:22.546314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb4c87-c97f-4864-94ea-ba4fbc483d90 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:22.563093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e68087061645019e1524959ef40957', 'request_id': 'req-0b9fe87a-1f3f-43f5-baff-3cba2508b988'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:22.594865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e68087061645019e1524959ef40957', 'request_id': 'req-0b9fe87a-1f3f-43f5-baff-3cba2508b988'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:22.613165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e68087061645019e1524959ef40957', 'request_id': 'req-0b9fe87a-1f3f-43f5-baff-3cba2508b988'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:22.660973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:22.661882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:22.662170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:22.662503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] PATH_INFO: `/v3/projects/5515c32dd4e344d9b5c7fb0d87eea0a8/users/12fe75d799334505b3df357af913f8ad/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:22.687858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5515c32dd4e344d9b5c7fb0d87eea0a8, user_id=12fe75d799334505b3df357af913f8ad, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:22.691099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:22.715552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] The token cache is being invalidate because project 90e68087061645019e1524959ef40957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:22.715552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:22.716733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0b9fe87a-1f3f-43f5-baff-3cba2508b988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:22.718653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 852/1691] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:22 2022] DELETE /identity/v3/projects/90e68087061645019e1524959ef40957 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:22.723946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:22.724238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:22.724514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:22.792854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] MFA Rules not processed for user `1c230254925a43ac9fe52dffbd0fe3ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:22.794501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:22.844526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 840/1692] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:22 2022] PUT /identity/v3/projects/5515c32dd4e344d9b5c7fb0d87eea0a8/users/12fe75d799334505b3df357af913f8ad/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:22.845642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:22.857281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:22.956478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 853/1693] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:22.957414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:22.962084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:23.004743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ded98-64c1-443a-b4b0-487bc07eaeec None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:23.098752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddedb76-9cd9-440c-abf9-6325efeb8458 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:23.117074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aba72b-9ab7-4408-8bf5-db3c59e7599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:23.117920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba72b-9ab7-4408-8bf5-db3c59e7599a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:23.118210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba72b-9ab7-4408-8bf5-db3c59e7599a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:23.118494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba72b-9ab7-4408-8bf5-db3c59e7599a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:23.119276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aba72b-9ab7-4408-8bf5-db3c59e7599a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:23.121142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aba72b-9ab7-4408-8bf5-db3c59e7599a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:23.140152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 841/1694] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:23.150822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd1ec0-b22d-483b-a44f-a89b04b1d43c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:23.152521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd1ec0-b22d-483b-a44f-a89b04b1d43c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:23.152521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd1ec0-b22d-483b-a44f-a89b04b1d43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:23.196933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:23.198818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:23.202477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:23.202894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] PATH_INFO: `/v3/users/f68422d627da4356803d5aae6c6f158d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:23.211193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f68422d627da4356803d5aae6c6f158d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:23.219966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:23.221624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-aacd1ec0-b22d-483b-a44f-a89b04b1d43c None None] MFA Rules not processed for user `12fe75d799334505b3df357af913f8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:23.222893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aacd1ec0-b22d-483b-a44f-a89b04b1d43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:23.440794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 842/1695] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:23.442137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:23.442431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:23.442706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:23.485209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f68422d627da4356803d5aae6c6f158d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:23.491107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:23.500309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f68422d627da4356803d5aae6c6f158d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:23.501481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f68422d627da4356803d5aae6c6f158d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:23.510655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f68422d627da4356803d5aae6c6f158d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:23.512030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f68422d627da4356803d5aae6c6f158d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:23.514303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] MFA Rules not processed for user `1c230254925a43ac9fe52dffbd0fe3ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:23.517861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:23.587771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:23.636658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 843/1696] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:23.637780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:23.643002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:23.655379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 854/1697] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:22 2022] DELETE /identity/v3/users/f68422d627da4356803d5aae6c6f158d => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:23.656535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:23.660034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:23.688507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfcf8f-a35e-4812-be50-d8289f30ef28 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:23.798676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:23.799611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:23.799906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:23.803155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] PATH_INFO: `/v3/projects/7a36c030f4ad4573b25fde06f7ea007d/users/b238ada9f514422d95625387bbda79fa/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:23.819711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81a4d-c1d0-4900-b5fe-f67537c7b14b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:23.835311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:23.836013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:23.836441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:23.836783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:23.838128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:23.845470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:23.849918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a36c030f4ad4573b25fde06f7ea007d, user_id=b238ada9f514422d95625387bbda79fa, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:23.854694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:23.941219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 844/1698] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:23 2022] PUT /identity/v3/projects/7a36c030f4ad4573b25fde06f7ea007d/users/b238ada9f514422d95625387bbda79fa/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:23.942272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:23.944127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:23.977862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 855/1699] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:23 2022] POST /identity/v3/projects => generated 355 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:23.977862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:24.136762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60f7f-1f6e-4d8f-9a15-63ab8391d736 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:24.159431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f98675-ccc7-4a94-915c-07ae62b02172 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:24.176358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:24.177402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:24.177842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:24.178289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:24.179345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:24.182360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:24.212358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 856/1700] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:24.212566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:24.236405 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55bcd4-be78-40bd-93cd-9355f57583f5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:24.253791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:24.254687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:24.255133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:24.256114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:24.257503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:24.261427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:24.284442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:24.287086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:24.287086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:24.287086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:24.303957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:24.436514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 857/1701] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:24 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:24.440515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:24.490501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a3cf3-ca50-431e-abf8-5a5793eb6a41 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:24.510169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:24.511604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:24.512068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:24.512490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] PATH_INFO: `/v3/projects/7a36c030f4ad4573b25fde06f7ea007d/users/b238ada9f514422d95625387bbda79fa/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:24.515832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a36c030f4ad4573b25fde06f7ea007d, user_id=b238ada9f514422d95625387bbda79fa, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:24.523361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:24.555911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 858/1702] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:24 2022] PUT /identity/v3/projects/7a36c030f4ad4573b25fde06f7ea007d/users/b238ada9f514422d95625387bbda79fa/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:24.557859 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:24.635157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:24.669951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2e026-17de-40c8-8dde-914af5a6c96f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:24.675408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77020154-34bc-4b1b-a5b9-2d30e83da02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:24.683008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77020154-34bc-4b1b-a5b9-2d30e83da02e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:24.683008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77020154-34bc-4b1b-a5b9-2d30e83da02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:24.683008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77020154-34bc-4b1b-a5b9-2d30e83da02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:24.683008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77020154-34bc-4b1b-a5b9-2d30e83da02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:24.683008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77020154-34bc-4b1b-a5b9-2d30e83da02e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:24.699740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 859/1703] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:24.704073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:24.706375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:24.706891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:24.754653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:24.757945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] MFA Rules not processed for user `b238ada9f514422d95625387bbda79fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:24.760639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:24.768150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:24.792230 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 845/1704] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:24.801700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:24.843988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38faea0-b389-40cb-835d-efce248a2301 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:24.864890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:24.866129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:24.866417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:24.866699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] PATH_INFO: `/v3/projects/4729acc4b8d04396a0762d0d36535c1b/users/06461bc41da84bc4a047d7061aa31ecb/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:24.928810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 860/1705] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:24.940121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:24.945046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:24.952320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4729acc4b8d04396a0762d0d36535c1b, user_id=06461bc41da84bc4a047d7061aa31ecb, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:24.965109 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:25.078219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660c617-40ec-4eb1-b3d9-fe1999a008e1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:25.084446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:25.085175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:25.085463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:25.085894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:25.102258 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:25.103764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:25.127822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 846/1707] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:24 2022] PUT /identity/v3/projects/4729acc4b8d04396a0762d0d36535c1b/users/06461bc41da84bc4a047d7061aa31ecb/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:25.128072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 861/1707] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:24 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:25.129715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:25.280698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:25.282935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:25.352152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fec99-a5f4-436f-9e72-ed62bd02dbb6 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:25.391385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f13e-fe13-41f1-960d-786ecaf50d7a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:25.395984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:25.400466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:25.403986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:25.405466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] PATH_INFO: `/v3/projects/86e6c7dd14994aaa9d6ca68d1a0e7713` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:25.407865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:25.408556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:25.408950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:25.409224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:25.410284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:25.411691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:25.414142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e6c7dd14994aaa9d6ca68d1a0e7713)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:25.424625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:25.430063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 847/1708] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:47:25.443179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:25.471863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:25.485438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea0e32-d2dd-4de5-a316-eefa6577ac54 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:25.509578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:25.516745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:25.516745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:25.516745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] PATH_INFO: `/v3/projects/4729acc4b8d04396a0762d0d36535c1b/users/06461bc41da84bc4a047d7061aa31ecb/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:25.531943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4729acc4b8d04396a0762d0d36535c1b, user_id=06461bc41da84bc4a047d7061aa31ecb, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:25.545568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:25.578723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:25.589831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 848/1709] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:25 2022] PUT /identity/v3/projects/4729acc4b8d04396a0762d0d36535c1b/users/06461bc41da84bc4a047d7061aa31ecb/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:25.593185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:25.812214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9e291-73a8-4d09-8993-17c52f5011ab admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:25.816013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e6c7dd14994aaa9d6ca68d1a0e7713', 'request_id': 'req-05de6120-c6bc-45e0-98fc-e10a84f4ca38'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:25.847910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e6c7dd14994aaa9d6ca68d1a0e7713', 'request_id': 'req-05de6120-c6bc-45e0-98fc-e10a84f4ca38'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:25.868773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e6c7dd14994aaa9d6ca68d1a0e7713', 'request_id': 'req-05de6120-c6bc-45e0-98fc-e10a84f4ca38'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:25.890077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:25.891211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:25.891880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:25.892297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] PATH_INFO: `/v3/users/2c65e34f5aad4f468e1706a5790bb274` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:25.893788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c65e34f5aad4f468e1706a5790bb274)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:25.896167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:25.999477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] The token cache is being invalidate because project 86e6c7dd14994aaa9d6ca68d1a0e7713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:26.000007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:26.001212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-05de6120-c6bc-45e0-98fc-e10a84f4ca38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:26.009055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 862/1710] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:25 2022] DELETE /identity/v3/projects/86e6c7dd14994aaa9d6ca68d1a0e7713 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:26.016163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0740a-32e0-4779-969f-381901e688f5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:26.016761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0740a-32e0-4779-969f-381901e688f5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:26.017262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0740a-32e0-4779-969f-381901e688f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:26.124001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9bc0740a-32e0-4779-969f-381901e688f5 None None] MFA Rules not processed for user `06461bc41da84bc4a047d7061aa31ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:26.126032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9bc0740a-32e0-4779-969f-381901e688f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:26.215196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c65e34f5aad4f468e1706a5790bb274'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:26.239074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c65e34f5aad4f468e1706a5790bb274'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:26.335208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c65e34f5aad4f468e1706a5790bb274'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:26.335929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c65e34f5aad4f468e1706a5790bb274'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:26.336706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e3c5e0a0-be5c-4687-83fa-f64a5e431c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c65e34f5aad4f468e1706a5790bb274'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:26.386663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 849/1711] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:25 2022] DELETE /identity/v3/users/2c65e34f5aad4f468e1706a5790bb274 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:26.391434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:26.391862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:26.395310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:26.450424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:26.452193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:26.490565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 863/1712] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:26 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:26.493737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada46da-9af6-4f2f-a339-27ff7c202029 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:26.497967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada46da-9af6-4f2f-a339-27ff7c202029 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:26.503993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada46da-9af6-4f2f-a339-27ff7c202029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:26.559751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3ada46da-9af6-4f2f-a339-27ff7c202029 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:26.562926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3ada46da-9af6-4f2f-a339-27ff7c202029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:26.721078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 850/1713] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:26.722403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:26.730055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:26.738120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 864/1714] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:26.740138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38bfcd-e3c9-4c11-8ba7-96876c7ed337 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:26.741582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38bfcd-e3c9-4c11-8ba7-96876c7ed337 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:26.742220 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38bfcd-e3c9-4c11-8ba7-96876c7ed337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:26.764914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:26.809660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cf38bfcd-e3c9-4c11-8ba7-96876c7ed337 None None] MFA Rules not processed for user `8873b1c9f58a4adea93ebe21f5f3263d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:26.811054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cf38bfcd-e3c9-4c11-8ba7-96876c7ed337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:26.822669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:26.967704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 865/1715] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:26.967704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:26.967704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:26.967704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:26.982394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd76b8-e29b-4e57-82d6-1fc1d5b43958 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:27.002325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44971dd9-f465-4494-ac58-e0ca514d597b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:27.003057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971dd9-f465-4494-ac58-e0ca514d597b service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.003338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971dd9-f465-4494-ac58-e0ca514d597b service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.003645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971dd9-f465-4494-ac58-e0ca514d597b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.018407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:27.021099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:27.050722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 866/1716] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:27.054235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:27.060360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:27.181690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44971dd9-f465-4494-ac58-e0ca514d597b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:27.183186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44971dd9-f465-4494-ac58-e0ca514d597b service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:27.199426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 851/1717] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:26 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:27.201120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93e07-8b1e-4a92-abdb-2e63724f330c None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.201576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93e07-8b1e-4a92-abdb-2e63724f330c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.207589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93e07-8b1e-4a92-abdb-2e63724f330c None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.208536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 852/1718] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 15 17:47:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:27.208853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f776a-a147-4658-a439-8f3bd729565d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:27.210457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c3666-90de-4b30-ac22-d9655d861827 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.210763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c3666-90de-4b30-ac22-d9655d861827 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.211166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c3666-90de-4b30-ac22-d9655d861827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.221403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:27.222085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.222371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.222655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.238906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-856c3666-90de-4b30-ac22-d9655d861827 None None] MFA Rules not processed for user `8873b1c9f58a4adea93ebe21f5f3263d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:27.245324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:27.247169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:27.248409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-856c3666-90de-4b30-ac22-d9655d861827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:27.257570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 853/1719] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:27.259497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.265117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.265117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.271086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 867/1720] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:27.271959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:27.292897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116aef38-5247-4f91-8ee0-efb5fa3929f3 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:27.305385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300dea9-578c-433d-b424-b478f1aaeb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:27.306075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300dea9-578c-433d-b424-b478f1aaeb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.306900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300dea9-578c-433d-b424-b478f1aaeb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.307182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300dea9-578c-433d-b424-b478f1aaeb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.308909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300dea9-578c-433d-b424-b478f1aaeb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:27.308909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300dea9-578c-433d-b424-b478f1aaeb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:27.309904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300dea9-578c-433d-b424-b478f1aaeb0a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:27.312295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] MFA Rules not processed for user `caab1412244c43adac226cb253a10880`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:27.314795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:27.365833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 868/1721] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:27.365833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee778f-7bc0-4d99-8230-80f5c4279f21 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.365833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee778f-7bc0-4d99-8230-80f5c4279f21 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.365833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee778f-7bc0-4d99-8230-80f5c4279f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.465637 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 854/1722] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:27.467184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:27.470289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-09ee778f-7bc0-4d99-8230-80f5c4279f21 None None] MFA Rules not processed for user `35d3862fe9e846f18d8bc3a0fa5e993a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:27.471536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-09ee778f-7bc0-4d99-8230-80f5c4279f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:27.499439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 869/1723] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:47:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:27.501217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee778f-7bc0-4d99-8230-80f5c4279f21 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:27.516578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:27.533254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee778f-7bc0-4d99-8230-80f5c4279f21 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:27.541516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:27.543458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.543956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.544433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.545393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:27.548022 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:27.596433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:27.665467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:27.674968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 870/1724] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:27 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:27.680245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:27.772971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16017b99-9c9f-4295-91a1-327a225b8c24 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:27.864377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a129ef-0539-433c-84c1-9418bcb837a8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:27.870922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:27.871872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.872340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.872816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.875053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:27.964160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dec788-6376-4b33-bba0-77c7438affa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:27.964663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec788-6376-4b33-bba0-77c7438affa5 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:27.964956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec788-6376-4b33-bba0-77c7438affa5 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:27.965240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec788-6376-4b33-bba0-77c7438affa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:27.971495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:27.973281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:27.984317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 871/1725] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:27 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:27.993549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:28.082694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dec788-6376-4b33-bba0-77c7438affa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:28.084365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dec788-6376-4b33-bba0-77c7438affa5 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:28.115925 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 855/1726] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:28.126734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:28.127038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:28.127539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:28.137615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b91d2-c079-4a55-b79f-7ac9026c06a7 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:28.143236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:28.144245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:28.144973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:28.145381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:28.166828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:28.173944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:28.187665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 872/1727] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:28.189104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:28.217722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] MFA Rules not processed for user `34e7019b46d743a1b6b0d3315e69053f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:28.218727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:28.392474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410bdc3-e7fa-4132-8b8f-6834fdc10406 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:28.412305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 856/1728] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:28.418904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:28.428744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:28.429669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:28.430117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:28.434641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:28.437978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:28.438886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:28.441356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:28.463538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e35a4-3f0a-454f-97eb-2c992cce901d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:28.474353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:28.474937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:28.475228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:28.475508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] PATH_INFO: `/v3/projects/2a6f8eb5152e4991bcc2d893b5c062be` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:28.477676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6f8eb5152e4991bcc2d893b5c062be)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:28.481923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:28.604857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 873/1729] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:28 2022] POST /identity/v3/users => generated 289 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:28.609469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:28.626577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36bb4c-e5be-4809-a7c3-785e4d323622 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:28.661914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:28.666959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:28.668275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:28.670376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:28.671524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:28.680168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:28.699542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6f8eb5152e4991bcc2d893b5c062be', 'request_id': 'req-6565ed30-016b-45fd-9a47-091fa4bc18e4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:28.724841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 874/1730] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:28.726166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:28.733351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6f8eb5152e4991bcc2d893b5c062be', 'request_id': 'req-6565ed30-016b-45fd-9a47-091fa4bc18e4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:28.760042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9071cf0-4156-4cef-a84a-421b04da6c2d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:28.847951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6f8eb5152e4991bcc2d893b5c062be', 'request_id': 'req-6565ed30-016b-45fd-9a47-091fa4bc18e4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:28.865954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] The token cache is being invalidate because project 2a6f8eb5152e4991bcc2d893b5c062be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:28.866251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:28.872287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6565ed30-016b-45fd-9a47-091fa4bc18e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:28.879117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 857/1731] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:28 2022] DELETE /identity/v3/projects/2a6f8eb5152e4991bcc2d893b5c062be => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:28.879117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:28.879117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:28.879117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:28.913739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:28.914516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:28.914874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:28.915225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:28.916470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:28.920579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:28.960992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] MFA Rules not processed for user `5013f4ba5cef44f7bdab6531ec0b0836`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:28.962195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:28.968903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 875/1732] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:28.969811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:28.979067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:29.023462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578aa604-9f4f-40ca-b9ec-88103170fc26 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:29.054056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:29.055694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:29.056711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:29.057132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] PATH_INFO: `/v3/users/b238ada9f514422d95625387bbda79fa` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:29.060054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b238ada9f514422d95625387bbda79fa)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:29.062251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:29.218102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 858/1733] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:29.218618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:29.224459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:29.251350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b6b2e-532f-4e6f-b5e1-14f32671df35 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:29.271269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b968c7-604f-41ec-8a6b-332b1c6735e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:29.272209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b968c7-604f-41ec-8a6b-332b1c6735e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:29.272500 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b968c7-604f-41ec-8a6b-332b1c6735e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:29.272856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b968c7-604f-41ec-8a6b-332b1c6735e1 admin admin] PATH_INFO: `/v3/projects/c99c59917e6945e8acc1e4be9adc4338/users/70713fae613d44cfbf3ee0f10161841e/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:29.306465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b968c7-604f-41ec-8a6b-332b1c6735e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99c59917e6945e8acc1e4be9adc4338, user_id=70713fae613d44cfbf3ee0f10161841e, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:29.310222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b968c7-604f-41ec-8a6b-332b1c6735e1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:29.387063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b238ada9f514422d95625387bbda79fa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:29.413940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 859/1734] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:29 2022] PUT /identity/v3/projects/c99c59917e6945e8acc1e4be9adc4338/users/70713fae613d44cfbf3ee0f10161841e/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:29.416888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc101-0bee-4c1f-aa9d-e657c077898e None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:29.418174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc101-0bee-4c1f-aa9d-e657c077898e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:29.418535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc101-0bee-4c1f-aa9d-e657c077898e None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:29.426142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 860/1735] 10.222.0.57 () {54 vars in 912 bytes} [Fri Jul 15 17:47:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:29.427179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dc101-0bee-4c1f-aa9d-e657c077898e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:29.429647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c68dc101-0bee-4c1f-aa9d-e657c077898e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:29.438381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b238ada9f514422d95625387bbda79fa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:29.439236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b238ada9f514422d95625387bbda79fa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:29.471146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b238ada9f514422d95625387bbda79fa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:29.473419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-851485e4-26b6-4146-bd09-1dfedafeb321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b238ada9f514422d95625387bbda79fa'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:29.622554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dc101-0bee-4c1f-aa9d-e657c077898e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:29.624939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 876/1736] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:28 2022] DELETE /identity/v3/users/b238ada9f514422d95625387bbda79fa => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:29.626470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0b57c-61b8-4f57-931a-af8289a5f423 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:29.626867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0b57c-61b8-4f57-931a-af8289a5f423 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:29.627326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0b57c-61b8-4f57-931a-af8289a5f423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:29.663474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bae0b57c-61b8-4f57-931a-af8289a5f423 None None] MFA Rules not processed for user `70713fae613d44cfbf3ee0f10161841e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:29.676630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bae0b57c-61b8-4f57-931a-af8289a5f423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:29.750727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:29.751594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:29.752027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:29.752455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:29.756474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:29.758275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 877/1737] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:29.766399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:29.766870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:29.768698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:29.812041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] MFA Rules not processed for user `35d3862fe9e846f18d8bc3a0fa5e993a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:29.813988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:29.852292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:29.959775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:30.011263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 878/1738] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:47:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:30.012522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:30.014172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:30.015340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:30.015814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:30.045008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 861/1739] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:30.046132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:30.047611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:30.162908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af561e92-f71e-4a9b-8be2-52a21024969f service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:30.169636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:30.170398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:30.170761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:30.171099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:30.171984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:30.173946 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:30.204533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ac246-4fac-4dc2-bca7-b56ad53be588 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:30.214685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:30.215365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:30.215724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:30.215999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:30.218463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:30.246576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:30.248317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:30.262271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 879/1740] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:30.263231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:30.265779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:30.279092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8ba2-d98e-4408-91c3-0859a82e33ef service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:30.285037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:30.285744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:30.286862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:30.287149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:30.297002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:30.298568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:30.322090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 862/1741] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:30 2022] POST /identity/v3/projects => generated 379 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:30.322969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:30.326230 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:30.338733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 880/1742] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:30.342769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:30.414501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33299e-a60a-442b-a57a-02d711c4d56a service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:30.481664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd7c1-0ace-4084-90f5-a0cba51fc20b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:30.487764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:30.489042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:30.489479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:30.489913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] PATH_INFO: `/v3/projects/7a36c030f4ad4573b25fde06f7ea007d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:30.491436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a36c030f4ad4573b25fde06f7ea007d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:30.494118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:30.509456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:30.510354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:30.510790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:30.511177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:30.520933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:30.524058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:30.702953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a36c030f4ad4573b25fde06f7ea007d', 'request_id': 'req-900bfdba-1213-40d1-9ae0-e2892057ad79'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:30.723605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 881/1743] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:30 2022] POST /identity/v3/users => generated 290 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:30.730643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:30.731186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a36c030f4ad4573b25fde06f7ea007d', 'request_id': 'req-900bfdba-1213-40d1-9ae0-e2892057ad79'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:30.761434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27061b7a-fc83-475f-86c0-33613c01c7ed admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:30.840104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a36c030f4ad4573b25fde06f7ea007d', 'request_id': 'req-900bfdba-1213-40d1-9ae0-e2892057ad79'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:30.854076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] The token cache is being invalidate because project 7a36c030f4ad4573b25fde06f7ea007d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:30.854661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:30.857057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-900bfdba-1213-40d1-9ae0-e2892057ad79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:30.859906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 863/1744] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:30 2022] DELETE /identity/v3/projects/7a36c030f4ad4573b25fde06f7ea007d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:30.867439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e1049-665c-4ba4-b37b-432e9cd75d36 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:30.867819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e1049-665c-4ba4-b37b-432e9cd75d36 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:30.868176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e1049-665c-4ba4-b37b-432e9cd75d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:30.901834 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-db1e1049-665c-4ba4-b37b-432e9cd75d36 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:30.903519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db1e1049-665c-4ba4-b37b-432e9cd75d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:30.967954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:30.969539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:30.969847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:30.970128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:30.970973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:30.972585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:30.983570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:31.015870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 882/1745] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:31.016928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:31.022984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:31.045176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 864/1746] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:31.054305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81bcc3-552e-40d0-be3f-dda35a37fb3c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:31.057137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd765e-d65c-4812-8590-e49a62d1cbf1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.057137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd765e-d65c-4812-8590-e49a62d1cbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.057372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd765e-d65c-4812-8590-e49a62d1cbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.067239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:31.073526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.073526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.073526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.073526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:31.079704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:31.081563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-81bd765e-d65c-4812-8590-e49a62d1cbf1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:31.084682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-81bd765e-d65c-4812-8590-e49a62d1cbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:31.098499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 865/1747] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:31.102734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd765e-d65c-4812-8590-e49a62d1cbf1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:31.108703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 883/1748] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:31.113262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:31.117484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd765e-d65c-4812-8590-e49a62d1cbf1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:31.134006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:31.145613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.145915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.149588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.152366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:31.152809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:31.157911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:31.163054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4744-b13f-47c5-91d7-a0a3a83d5779 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:31.170234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a623f6-aa40-4024-b0b1-5ecad4827251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:31.171300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a623f6-aa40-4024-b0b1-5ecad4827251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.171934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a623f6-aa40-4024-b0b1-5ecad4827251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.172470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a623f6-aa40-4024-b0b1-5ecad4827251 admin admin] PATH_INFO: `/v3/projects/9ee41bec344d447f8e5ce0506bab29f7/users/6d18c2f1d834418295baf21516c1d701/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.194911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a623f6-aa40-4024-b0b1-5ecad4827251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee41bec344d447f8e5ce0506bab29f7, user_id=6d18c2f1d834418295baf21516c1d701, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:31.202368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a623f6-aa40-4024-b0b1-5ecad4827251 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:31.222852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 866/1749] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:47:31.227909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:31.254727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd672b-a55d-426c-b8bc-9ef4352387ad admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:31.276773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:31.277969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.278268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.278555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.279270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:31.281756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:31.299673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 884/1750] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:31 2022] PUT /identity/v3/projects/9ee41bec344d447f8e5ce0506bab29f7/users/6d18c2f1d834418295baf21516c1d701/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:47:31.311293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935765ba-3eeb-4f93-b83d-7b85a05b3ff4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.311595 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935765ba-3eeb-4f93-b83d-7b85a05b3ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.311883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935765ba-3eeb-4f93-b83d-7b85a05b3ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.418563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-935765ba-3eeb-4f93-b83d-7b85a05b3ff4 None None] MFA Rules not processed for user `6d18c2f1d834418295baf21516c1d701`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:31.420271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-935765ba-3eeb-4f93-b83d-7b85a05b3ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:31.516707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 867/1751] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:31 2022] POST /identity/v3/projects => generated 373 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:31.518269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:31.525819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 885/1752] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:31.530626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935765ba-3eeb-4f93-b83d-7b85a05b3ff4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:31.605737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935765ba-3eeb-4f93-b83d-7b85a05b3ff4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:31.672301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:31.673082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.673384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.673662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.674371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:31.676699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:31.805953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4311-9968-4b9e-8462-885bc18ae2ce admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:31.823855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:31.824553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:31.824930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:31.825215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:31.828553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:31.830610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:31.863640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 886/1753] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:31 2022] POST /identity/v3/projects => generated 377 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:31.866506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:31.996132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:32.004722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 868/1754] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:31 2022] POST /identity/v3/users => generated 287 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:32.005750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:32.027746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff130c-a816-4d08-ba51-2e891b6d9634 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:32.041760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:32.043564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.044098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.044571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.052298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:32.054110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:32.083395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 869/1755] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:47:32.087167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:32.089308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b9dd0-1b6e-48cf-b50a-9f93df63f43f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:32.106516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140c8db-c511-4c13-a34c-aa2c4ca4263a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:32.107890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:32.109768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.110059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.110339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.111273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:32.113903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20ff48-2cb2-45e0-b3d7-6f969bddb000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:32.114707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20ff48-2cb2-45e0-b3d7-6f969bddb000 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.115066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20ff48-2cb2-45e0-b3d7-6f969bddb000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.115413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20ff48-2cb2-45e0-b3d7-6f969bddb000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.116728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:32.116842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20ff48-2cb2-45e0-b3d7-6f969bddb000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:32.148942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20ff48-2cb2-45e0-b3d7-6f969bddb000 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:32.176404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 870/1756] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:47:32.178827 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29673c-cfc4-43c0-99ba-98b0b5680c2e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.183516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29673c-cfc4-43c0-99ba-98b0b5680c2e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.183516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29673c-cfc4-43c0-99ba-98b0b5680c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.234403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2d29673c-cfc4-43c0-99ba-98b0b5680c2e None None] MFA Rules not processed for user `8873b1c9f58a4adea93ebe21f5f3263d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:32.245383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2d29673c-cfc4-43c0-99ba-98b0b5680c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:32.267079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:32.274108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 887/1757] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:31 2022] POST /identity/v3/users => generated 295 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:32.274108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:32.323944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f7697-0817-4c97-af66-5a813bb1358b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:32.358401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ac772-d6e2-47f7-8b5c-9c3c2a5ba6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:32.359491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ac772-d6e2-47f7-8b5c-9c3c2a5ba6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.363336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ac772-d6e2-47f7-8b5c-9c3c2a5ba6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.363635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ac772-d6e2-47f7-8b5c-9c3c2a5ba6ec admin admin] PATH_INFO: `/v3/projects/234bd5a5ea18455eb476d5f01165b18f/users/b0fa7d2e028d49f9a2235c41fc1ffe77/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.394441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ac772-d6e2-47f7-8b5c-9c3c2a5ba6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=234bd5a5ea18455eb476d5f01165b18f, user_id=b0fa7d2e028d49f9a2235c41fc1ffe77, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:32.394441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ac772-d6e2-47f7-8b5c-9c3c2a5ba6ec admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:32.408215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 871/1758] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:32.408215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29673c-cfc4-43c0-99ba-98b0b5680c2e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:32.421564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29673c-cfc4-43c0-99ba-98b0b5680c2e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:32.427050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:32.428121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.428518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.428967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.433423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:32.436785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:32.456117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 872/1759] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:32.459462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:32.500267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e98091-5e1b-4ff1-8b92-29b987161492 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:32.516178 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:32.517409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.518751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.519194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] PATH_INFO: `/v3/projects/3349255c8e3e4cb98da1905248acd79b/users/6a35421930fd42b6a15bbe4e130fd29a/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.525992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 888/1760] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:32 2022] PUT /identity/v3/projects/234bd5a5ea18455eb476d5f01165b18f/users/b0fa7d2e028d49f9a2235c41fc1ffe77/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:32.543865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9535936-1688-4b14-8aa5-a201425401d2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:32.547167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9535936-1688-4b14-8aa5-a201425401d2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:32.550700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9535936-1688-4b14-8aa5-a201425401d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:32.610641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3349255c8e3e4cb98da1905248acd79b, user_id=6a35421930fd42b6a15bbe4e130fd29a, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:32.625196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a9535936-1688-4b14-8aa5-a201425401d2 None None] MFA Rules not processed for user `b0fa7d2e028d49f9a2235c41fc1ffe77`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:32.625427 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:32.626114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a9535936-1688-4b14-8aa5-a201425401d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:32.829778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 873/1761] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:32 2022] PUT /identity/v3/projects/3349255c8e3e4cb98da1905248acd79b/users/6a35421930fd42b6a15bbe4e130fd29a/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:32.834253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:32.839921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 889/1762] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:32.848511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9535936-1688-4b14-8aa5-a201425401d2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:33.073026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa95ba8-3262-493e-9258-7c1215ebb16f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.082012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9535936-1688-4b14-8aa5-a201425401d2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.097506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:33.102834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.104046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.104149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:33.104238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.104805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.105000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:33.107278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:33.107485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.107838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.110336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:33.114052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:33.129314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 874/1763] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:33.133905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:33.164418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba6dd-08eb-40d6-89e1-08f9c93c6b12 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.177789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719776c1-07c8-4619-a5f4-c0669499dbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:33.180931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719776c1-07c8-4619-a5f4-c0669499dbcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.181838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719776c1-07c8-4619-a5f4-c0669499dbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.182128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719776c1-07c8-4619-a5f4-c0669499dbcb admin admin] PATH_INFO: `/v3/projects/3349255c8e3e4cb98da1905248acd79b/users/6a35421930fd42b6a15bbe4e130fd29a/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.187612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719776c1-07c8-4619-a5f4-c0669499dbcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3349255c8e3e4cb98da1905248acd79b, user_id=6a35421930fd42b6a15bbe4e130fd29a, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:33.195356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719776c1-07c8-4619-a5f4-c0669499dbcb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:33.225709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 875/1764] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:33 2022] PUT /identity/v3/projects/3349255c8e3e4cb98da1905248acd79b/users/6a35421930fd42b6a15bbe4e130fd29a/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:33.232401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.232805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.233135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.241675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 890/1765] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:32 2022] POST /identity/v3/projects => generated 373 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:33.250516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:33.306507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] MFA Rules not processed for user `6a35421930fd42b6a15bbe4e130fd29a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:33.307772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:33.366033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeeca0-348b-46ba-855f-f6787ba5ca82 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.374428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:33.375384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.376009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.376493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.378061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:33.383306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:33.464649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 876/1766] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:33.478571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:33.480122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:33.551037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 891/1767] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:33 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:33.552650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:33.580864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fa0e2-c340-486c-91ad-f31e92193496 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.595244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:33.595982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.596273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.597584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.598540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:33.602376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:33.608564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea632229-f5df-45ea-bfe4-ce46f274eb36 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.614408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:33.615381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.615869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.616335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.632018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 892/1768] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:33.633000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:33.637680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:33.642828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:33.652915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed2eef-cde4-4bb1-9644-a93f30fb6866 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.667119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:33.668354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:33.668662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:33.668950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] PATH_INFO: `/v3/projects/5faf61c7105f462798841e6df7753ca7/users/1b3788f2a2844868a6607f452c84ee57/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:33.676022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 877/1769] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:33.742049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5faf61c7105f462798841e6df7753ca7, user_id=1b3788f2a2844868a6607f452c84ee57, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:33.745509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:33.853505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:33.856265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:33.876593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22869ae8-ab0a-476b-a1eb-ba98c6697737 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:33.883890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 893/1770] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:33 2022] PUT /identity/v3/projects/5faf61c7105f462798841e6df7753ca7/users/1b3788f2a2844868a6607f452c84ee57/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:33.887672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:34.060887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e539c-a7db-4fe3-818e-170a7a645c7c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:34.075265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:34.075963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:34.076252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:34.076537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:34.077354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:34.082861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:34.113127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:34.115404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:34.116491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:34.118358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:34.126878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:34.136885 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 894/1771] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:47:34.140914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:34.185627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d8982-2b0d-44e0-8de5-ec5084b4d570 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:34.203428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1fd80-3322-400d-9e57-c46d25f18600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:34.204352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1fd80-3322-400d-9e57-c46d25f18600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:34.204960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1fd80-3322-400d-9e57-c46d25f18600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:34.207061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1fd80-3322-400d-9e57-c46d25f18600 admin admin] PATH_INFO: `/v3/projects/5faf61c7105f462798841e6df7753ca7/users/1b3788f2a2844868a6607f452c84ee57/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:34.213097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1fd80-3322-400d-9e57-c46d25f18600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5faf61c7105f462798841e6df7753ca7, user_id=1b3788f2a2844868a6607f452c84ee57, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:34.216296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1fd80-3322-400d-9e57-c46d25f18600 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:34.249854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 895/1772] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:34 2022] PUT /identity/v3/projects/5faf61c7105f462798841e6df7753ca7/users/1b3788f2a2844868a6607f452c84ee57/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:47:34.262228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ad99-d2cc-4ad9-b386-3a96f5148bad None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:34.262780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ad99-d2cc-4ad9-b386-3a96f5148bad None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:34.267578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ad99-d2cc-4ad9-b386-3a96f5148bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:34.335487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:34.342413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:34.356851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5552ad99-d2cc-4ad9-b386-3a96f5148bad None None] MFA Rules not processed for user `1b3788f2a2844868a6607f452c84ee57`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:34.357329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 878/1773] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:34.358385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5552ad99-d2cc-4ad9-b386-3a96f5148bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:34.541959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 896/1774] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:34.571211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:34.584933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9fe1e-b364-449f-97a2-45bf4ad179e8 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:34.791280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75335c77-e791-44c8-a47d-edf663e078af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:34.792781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75335c77-e791-44c8-a47d-edf663e078af service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:34.793215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75335c77-e791-44c8-a47d-edf663e078af service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:34.793634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75335c77-e791-44c8-a47d-edf663e078af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:34.843697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75335c77-e791-44c8-a47d-edf663e078af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:34.851366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75335c77-e791-44c8-a47d-edf663e078af service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:34.880312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 879/1775] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:35.074593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:35.548323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ad99-d2cc-4ad9-b386-3a96f5148bad None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:35.569247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ad99-d2cc-4ad9-b386-3a96f5148bad None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:35.573247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eeb675-1144-4f55-b153-cf5b649ad450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:35.574975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb675-1144-4f55-b153-cf5b649ad450 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:35.575265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb675-1144-4f55-b153-cf5b649ad450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:35.575838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb675-1144-4f55-b153-cf5b649ad450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:35.748277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeb675-1144-4f55-b153-cf5b649ad450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:35.752627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:35.764101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eeb675-1144-4f55-b153-cf5b649ad450 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:35.796855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 897/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:36.215009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:36.216002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:36.216429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:36.242332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] MFA Rules not processed for user `caab1412244c43adac226cb253a10880`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:36.245870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:36.267647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe222df-0ad0-451a-80fe-d1d1e027385d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:36.267647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe222df-0ad0-451a-80fe-d1d1e027385d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:36.267936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe222df-0ad0-451a-80fe-d1d1e027385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:36.344963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 880/1777] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:36.410086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-4fe222df-0ad0-451a-80fe-d1d1e027385d None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:36.411509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4fe222df-0ad0-451a-80fe-d1d1e027385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:36.598821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 898/1778] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:47:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:37.117828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:37.121212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:37.316521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87df71-c793-4a26-ba71-d1e8cfae2f43 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:37.324184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:37.325161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:37.325595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:37.326016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:37.336572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:37.340380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:37.352940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 881/1779] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:47:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:47:38.572931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:38.572931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:38.572931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:38.624004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] MFA Rules not processed for user `35d3862fe9e846f18d8bc3a0fa5e993a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:38.624884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:38.759341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 899/1780] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Jul 15 17:47:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:38.842752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:38.845385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:38.846606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:38.873551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc4598-82d7-4a3f-9a5e-8e4d190e7024 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:38.883525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:38.885999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:38.898473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:38.898473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:38.898473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:38.898473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:38.904544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66092ca-0ad6-4be9-bd7d-4c2aa37e62a5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:38.927588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a391b-c131-4b7f-810f-a4eb307be5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:38.928842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a391b-c131-4b7f-810f-a4eb307be5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:38.929186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a391b-c131-4b7f-810f-a4eb307be5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:38.929471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a391b-c131-4b7f-810f-a4eb307be5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:38.933530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:38.935513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:38.950347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a391b-c131-4b7f-810f-a4eb307be5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:38.955510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a391b-c131-4b7f-810f-a4eb307be5d7 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:38.974026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 882/1781] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:38.998952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 900/1782] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:39.673450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:40.353193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:40.589986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668941d7-7f2a-4cd5-892a-1068f983dbfe service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:40.595541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:40.603354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:40.603354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:40.603354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] PATH_INFO: `/v3/users/70713fae613d44cfbf3ee0f10161841e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:40.603354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=70713fae613d44cfbf3ee0f10161841e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:40.603354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:40.912543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70713fae613d44cfbf3ee0f10161841e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:40.927968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70713fae613d44cfbf3ee0f10161841e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:41.057393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70713fae613d44cfbf3ee0f10161841e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:41.058149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70713fae613d44cfbf3ee0f10161841e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:41.058851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70713fae613d44cfbf3ee0f10161841e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:41.125632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 883/1783] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:40 2022] DELETE /identity/v3/users/70713fae613d44cfbf3ee0f10161841e => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:41.130940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5ac85-c265-41f5-98e2-625aa9012ff0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:41.133056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5ac85-c265-41f5-98e2-625aa9012ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:41.134630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5ac85-c265-41f5-98e2-625aa9012ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:41.163108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fed5ac85-c265-41f5-98e2-625aa9012ff0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:41.174022 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fed5ac85-c265-41f5-98e2-625aa9012ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:41.336510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 901/1784] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:41.350387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:41.351895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:41.494626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd864f15-036b-42c0-a7a5-158fdb345694 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:41.496078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd864f15-036b-42c0-a7a5-158fdb345694 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:41.496527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd864f15-036b-42c0-a7a5-158fdb345694 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:41.500590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 902/1785] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:47:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:41.541208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b228-952e-46db-a5b2-34b8299c97ab admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:41.555397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:41.562024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:41.562585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:41.564235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:41.573247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:41.573883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:41.574285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:41.597443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:41.599119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:41.631760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 884/1786] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:47:41.660312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:41.662872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:41.823402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 903/1787] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:47:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:41.853779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:41.856692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:41.883436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab29ae9-a1b1-4e61-8253-37626d37d831 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:41.908362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:41.910603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:41.916585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:41.917093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:41.937622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:41.939417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:41.958059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 885/1788] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:42.557936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:42.724293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:42.725917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:42.755059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94ed6c-99fb-4114-bf0a-c5bedcd8ea1a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:42.769993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:42.769993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:42.769993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:42.769993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] PATH_INFO: `/v3/projects/c99c59917e6945e8acc1e4be9adc4338` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:42.770318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99c59917e6945e8acc1e4be9adc4338)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:42.771730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:42.951435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99c59917e6945e8acc1e4be9adc4338', 'request_id': 'req-bde74518-3334-4961-97e6-17c1646c99c7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:42.960017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99c59917e6945e8acc1e4be9adc4338', 'request_id': 'req-bde74518-3334-4961-97e6-17c1646c99c7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:42.972823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99c59917e6945e8acc1e4be9adc4338', 'request_id': 'req-bde74518-3334-4961-97e6-17c1646c99c7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.060040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] The token cache is being invalidate because project c99c59917e6945e8acc1e4be9adc4338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:43.060040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.060040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bde74518-3334-4961-97e6-17c1646c99c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.060040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 904/1789] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:42 2022] DELETE /identity/v3/projects/c99c59917e6945e8acc1e4be9adc4338 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:43.062215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:43.079359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0f611-7280-4a5e-803d-69483a268bac service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:43.095654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651edfe-bef4-46c9-bd23-5452939ab816 None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:43.095942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651edfe-bef4-46c9-bd23-5452939ab816 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:43.096521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651edfe-bef4-46c9-bd23-5452939ab816 None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:43.098325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 905/1790] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:47:43.126798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:43.128708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:43.128708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:43.128708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:43.198452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:43.199997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:43.220957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:43.222515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:43.223552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:43.229275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] PATH_INFO: `/v3/users/6d18c2f1d834418295baf21516c1d701` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:43.234316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d18c2f1d834418295baf21516c1d701)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:43.236325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:43.392876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 906/1791] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:47:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:43.604214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:43.604214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:43.620588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd638c3-d94e-47ad-8725-2acb9a828eda None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:43.625228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d18c2f1d834418295baf21516c1d701'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.627355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:43.628391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:43.628932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:43.629368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:43.637387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d18c2f1d834418295baf21516c1d701'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.642082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:43.643752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:43.656982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 907/1792] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:43.729992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d18c2f1d834418295baf21516c1d701'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.730469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d18c2f1d834418295baf21516c1d701'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.731195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d18c2f1d834418295baf21516c1d701'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:43.770369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 886/1793] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:43 2022] DELETE /identity/v3/users/6d18c2f1d834418295baf21516c1d701 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:44.340443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:44.349013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:44.379203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c6ff4-43c4-4042-9cd3-9375565f2ed0 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:44.495637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:44.581184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:44.582211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:44.582610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:44.583033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] PATH_INFO: `/v3/projects/9ee41bec344d447f8e5ce0506bab29f7` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:44.585979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee41bec344d447f8e5ce0506bab29f7)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:44.591892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:44.814072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee41bec344d447f8e5ce0506bab29f7', 'request_id': 'req-b2a57989-0dc1-4304-b7e5-126917397be4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:44.838615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee41bec344d447f8e5ce0506bab29f7', 'request_id': 'req-b2a57989-0dc1-4304-b7e5-126917397be4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:44.856499 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee41bec344d447f8e5ce0506bab29f7', 'request_id': 'req-b2a57989-0dc1-4304-b7e5-126917397be4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:44.953411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] The token cache is being invalidate because project 9ee41bec344d447f8e5ce0506bab29f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:44.956725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:44.956725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:44.960156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 908/1794] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:44 2022] DELETE /identity/v3/projects/9ee41bec344d447f8e5ce0506bab29f7 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:44.961638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:44.968883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:45.041162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:45.052046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1e900-68a3-466d-af6b-08b067017cc3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:45.205923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:45.207114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:45.207409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:45.208371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] PATH_INFO: `/v3/users/6a35421930fd42b6a15bbe4e130fd29a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:45.210483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a35421930fd42b6a15bbe4e130fd29a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:45.213304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:45.579794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a35421930fd42b6a15bbe4e130fd29a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:45.604718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a35421930fd42b6a15bbe4e130fd29a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:45.772693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a35421930fd42b6a15bbe4e130fd29a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:45.773605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a35421930fd42b6a15bbe4e130fd29a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:45.775068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c70a2f6a-61ba-484c-adc0-8c77903e8129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a35421930fd42b6a15bbe4e130fd29a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:45.830663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 887/1795] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:44 2022] DELETE /identity/v3/users/6a35421930fd42b6a15bbe4e130fd29a => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:46.987215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:47.564977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:47.567190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:47.591051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a57989-0dc1-4304-b7e5-126917397be4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:47.690624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:47.852145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:47.854254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:47.854254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:47.854254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] PATH_INFO: `/v3/projects/3349255c8e3e4cb98da1905248acd79b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:47.856939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3349255c8e3e4cb98da1905248acd79b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:47.867346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:48.088284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3349255c8e3e4cb98da1905248acd79b', 'request_id': 'req-d62a021f-1247-4082-8219-ab9dabd5a00d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:48.115978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3349255c8e3e4cb98da1905248acd79b', 'request_id': 'req-d62a021f-1247-4082-8219-ab9dabd5a00d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:48.123948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3349255c8e3e4cb98da1905248acd79b', 'request_id': 'req-d62a021f-1247-4082-8219-ab9dabd5a00d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:48.267097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] The token cache is being invalidate because project 3349255c8e3e4cb98da1905248acd79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:48.267097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:48.267097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d62a021f-1247-4082-8219-ab9dabd5a00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:48.267097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 909/1796] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:47 2022] DELETE /identity/v3/projects/3349255c8e3e4cb98da1905248acd79b => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:48.268281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047286-582d-44ff-a841-1bd892b698d1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:48.268281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047286-582d-44ff-a841-1bd892b698d1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:48.268281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047286-582d-44ff-a841-1bd892b698d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:48.318148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-db047286-582d-44ff-a841-1bd892b698d1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:48.320267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db047286-582d-44ff-a841-1bd892b698d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:48.583673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 888/1797] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:48.585449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843b23-a1f3-4543-a9e9-b5713e1e48ef None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:48.585974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843b23-a1f3-4543-a9e9-b5713e1e48ef None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:48.586416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843b23-a1f3-4543-a9e9-b5713e1e48ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:48.625230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a5843b23-a1f3-4543-a9e9-b5713e1e48ef None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:48.645415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a5843b23-a1f3-4543-a9e9-b5713e1e48ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:48.682425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 910/1798] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:48.687011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047286-582d-44ff-a841-1bd892b698d1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:48.716698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047286-582d-44ff-a841-1bd892b698d1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:48.739468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:48.740134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:48.740447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:48.740735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:48.741494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:48.741866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:48.744286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:48.775202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 889/1799] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:48.775947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5843b23-a1f3-4543-a9e9-b5713e1e48ef None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:48.789183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5843b23-a1f3-4543-a9e9-b5713e1e48ef None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:48.827583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:48.827583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:48.827583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:48.827583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:48.827910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:48.827910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:48.867086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:49.009434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 911/1800] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:48 2022] POST /identity/v3/projects => generated 371 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:47:49.009750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:49.031543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd358d52-1425-44ec-b97f-4053ef24acf8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:49.222787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:49.224235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:49.224570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:49.225060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:49.226197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:49.230525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:49.416479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 890/1801] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:49 2022] POST /identity/v3/users => generated 286 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:47:49.417688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:49.439427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3db9d2-509a-4841-a010-ef17be139796 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:49.460821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:49.460821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:49.460821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:49.460821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:49.473164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:49.477965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:49.516383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 912/1802] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:49.517221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:49.530126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f41998-543a-4ae2-9325-aee8532ac55d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:49.582729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba320813-72a9-470e-8917-1242ae52ff38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:49.583811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba320813-72a9-470e-8917-1242ae52ff38 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:49.584285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba320813-72a9-470e-8917-1242ae52ff38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:49.585014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba320813-72a9-470e-8917-1242ae52ff38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:49.586413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba320813-72a9-470e-8917-1242ae52ff38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:49.590553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba320813-72a9-470e-8917-1242ae52ff38 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:49.634773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 891/1803] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:49.637947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:49.657177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfc7b-2695-4459-bf35-2347afdc2020 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:49.668111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:49.672696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:49.672696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:49.672696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] PATH_INFO: `/v3/projects/a8064e0a94be4b918b133154eb1aefa4/users/48eeb6c70cf8439c9a23d9f3d2e4a3ea/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:49.693458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8064e0a94be4b918b133154eb1aefa4, user_id=48eeb6c70cf8439c9a23d9f3d2e4a3ea, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:49.697330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:49.731199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:49.823268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 913/1804] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:49 2022] PUT /identity/v3/projects/a8064e0a94be4b918b133154eb1aefa4/users/48eeb6c70cf8439c9a23d9f3d2e4a3ea/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:49.824876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72bc92-c830-42ce-b6f5-fca62aa719d4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:49.825402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72bc92-c830-42ce-b6f5-fca62aa719d4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:49.825491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72bc92-c830-42ce-b6f5-fca62aa719d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:49.885471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1e72bc92-c830-42ce-b6f5-fca62aa719d4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:49.889763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1e72bc92-c830-42ce-b6f5-fca62aa719d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:50.009780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 892/1805] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:50.071070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:50.074050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:50.349174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230d239-a24d-49fd-a85e-6530001e140a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:50.375986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:50.385581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:50.385581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:50.385581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:50.410926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:50.414245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:50.457536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 914/1806] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:51.189813 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:51.509237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:51.823474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:54.648560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e954c-b129-40d1-9e87-f521e0b9ae0f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:54.649139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e954c-b129-40d1-9e87-f521e0b9ae0f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:54.649237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e954c-b129-40d1-9e87-f521e0b9ae0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:54.704378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c36e954c-b129-40d1-9e87-f521e0b9ae0f None None] MFA Rules not processed for user `48eeb6c70cf8439c9a23d9f3d2e4a3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:54.704378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c36e954c-b129-40d1-9e87-f521e0b9ae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:54.734703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:54.734703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:54.753709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4b7e2-d257-4f11-a8c0-e6b6c9d032dd service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:54.760753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:54.761652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:54.761937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:54.762332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] PATH_INFO: `/v3/users/12fe75d799334505b3df357af913f8ad` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:54.764472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=12fe75d799334505b3df357af913f8ad)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:54.769183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:54.807265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 893/1807] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:54.811539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e954c-b129-40d1-9e87-f521e0b9ae0f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:54.821723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e954c-b129-40d1-9e87-f521e0b9ae0f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:54.826341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:54.828746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:54.828746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:54.828746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:54.828746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:54.830491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:54.986629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 894/1808] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:54 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:47:54.990370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:55.051717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d1a66-0753-4f65-ab30-271ed93b7d41 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:55.052241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12fe75d799334505b3df357af913f8ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:55.056636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:55.057306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:55.057596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:55.057936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:55.058645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:55.060628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:55.064790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12fe75d799334505b3df357af913f8ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:55.065496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12fe75d799334505b3df357af913f8ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:55.072658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12fe75d799334505b3df357af913f8ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:55.073298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-493daee0-873e-411a-b65e-f3a246bb64bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12fe75d799334505b3df357af913f8ad'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:55.216361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 915/1809] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:54 2022] DELETE /identity/v3/users/12fe75d799334505b3df357af913f8ad => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:47:55.219336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 895/1810] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:54 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:47:55.221195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:55.221696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:55.221827 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:55.222009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:55.223532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:55.236852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b880b-9466-4399-a280-c060ba23c7e6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:55.250231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:55.251469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:55.267848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:55.310530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:55.311216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:55.311506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:55.311784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:55.312519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:55.314355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:55.317722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 916/1811] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:55.324867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 896/1812] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:47:55.328101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:55.329706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:55.330983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:55.344978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409924d0-03d3-4be4-ac84-f5185f9a81f7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:55.346511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:55.347462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:55.347749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:55.348132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] PATH_INFO: `/v3/projects/5bac4a8fd2a1431892c25a6467d00ffd/users/2a607ebdf7354405bedc9f546416cb39/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:55.367116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bac4a8fd2a1431892c25a6467d00ffd, user_id=2a607ebdf7354405bedc9f546416cb39, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:55.369989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:55.398337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eac14b-7bce-45b1-9a45-a0299e462e25 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:55.402568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:55.403217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:55.403508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:55.403786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:55.413693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:55.415161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:55.425149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 917/1813] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:47:55.453914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 897/1814] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:55 2022] PUT /identity/v3/projects/5bac4a8fd2a1431892c25a6467d00ffd/users/2a607ebdf7354405bedc9f546416cb39/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:47:55.455623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:55.463035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:55.468824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c52357-2174-4991-8a31-6ea87f6270d3 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:55.530828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:55.539912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:55.539912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:55.539912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:55.539912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:55.539912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:55.552037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 918/1815] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:55.553495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:55.565066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec4b7-20ec-4cd1-935b-f4491876e934 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:55.570365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:55.571702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:55.572089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:55.572474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] PATH_INFO: `/v3/projects/5bac4a8fd2a1431892c25a6467d00ffd/users/2a607ebdf7354405bedc9f546416cb39/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:55.574787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bac4a8fd2a1431892c25a6467d00ffd, user_id=2a607ebdf7354405bedc9f546416cb39, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:55.579856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:55.590355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 898/1816] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:55 2022] PUT /identity/v3/projects/5bac4a8fd2a1431892c25a6467d00ffd/users/2a607ebdf7354405bedc9f546416cb39/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:55.920523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:55.926567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:55.944522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783303f-8377-4de8-9c2f-4bca81dc0312 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:56.013443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:56.014464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:56.014760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:56.015325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] PATH_INFO: `/v3/projects/5515c32dd4e344d9b5c7fb0d87eea0a8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:56.016286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5515c32dd4e344d9b5c7fb0d87eea0a8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:56.018385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:56.159059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5515c32dd4e344d9b5c7fb0d87eea0a8', 'request_id': 'req-d3c49606-f670-42c1-aea9-d78caf26441e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.165957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5515c32dd4e344d9b5c7fb0d87eea0a8', 'request_id': 'req-d3c49606-f670-42c1-aea9-d78caf26441e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.171564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5515c32dd4e344d9b5c7fb0d87eea0a8', 'request_id': 'req-d3c49606-f670-42c1-aea9-d78caf26441e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.290989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] The token cache is being invalidate because project 5515c32dd4e344d9b5c7fb0d87eea0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:56.291281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.301128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.303153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 919/1817] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:55 2022] DELETE /identity/v3/projects/5515c32dd4e344d9b5c7fb0d87eea0a8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:56.307695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:56.309598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:56.328045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fe891-55cb-48d3-ad89-002d6fc7a587 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:56.394032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:56.395092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:56.395472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:56.395850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] PATH_INFO: `/v3/users/06461bc41da84bc4a047d7061aa31ecb` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:56.397435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06461bc41da84bc4a047d7061aa31ecb)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:56.399513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:56.602218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06461bc41da84bc4a047d7061aa31ecb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.610355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06461bc41da84bc4a047d7061aa31ecb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.693563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06461bc41da84bc4a047d7061aa31ecb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.695465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06461bc41da84bc4a047d7061aa31ecb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.695465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4967096-3127-47f2-9f50-c5f424a14358 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06461bc41da84bc4a047d7061aa31ecb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:56.719912 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 899/1818] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:47:56 2022] DELETE /identity/v3/users/06461bc41da84bc4a047d7061aa31ecb => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:57.381976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:57.392866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c49606-f670-42c1-aea9-d78caf26441e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:57.452507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:57.453416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:57.453736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:57.454016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] PATH_INFO: `/v3/projects/4729acc4b8d04396a0762d0d36535c1b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:57.455214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4729acc4b8d04396a0762d0d36535c1b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:57.457275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:57.475034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:57.475502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:57.475774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:57.509550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] MFA Rules not processed for user `2a607ebdf7354405bedc9f546416cb39`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:57.510835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:57.576107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4729acc4b8d04396a0762d0d36535c1b', 'request_id': 'req-152af852-b1a1-4302-80c9-4e10b077a5e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:57.579746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 900/1819] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:57.582517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4729acc4b8d04396a0762d0d36535c1b', 'request_id': 'req-152af852-b1a1-4302-80c9-4e10b077a5e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:57.588427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4729acc4b8d04396a0762d0d36535c1b', 'request_id': 'req-152af852-b1a1-4302-80c9-4e10b077a5e4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:57.592365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:57.593779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:57.650749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21e59a-5c03-4b84-9be4-a678cb013603 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:57.677978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] The token cache is being invalidate because project 4729acc4b8d04396a0762d0d36535c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:47:57.678282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:57.681989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-152af852-b1a1-4302-80c9-4e10b077a5e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:47:57.681989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 920/1820] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:47:57 2022] DELETE /identity/v3/projects/4729acc4b8d04396a0762d0d36535c1b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:57.686070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec85d7d-13a8-42cb-9ad5-ec91da12c897 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:57.686366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec85d7d-13a8-42cb-9ad5-ec91da12c897 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:57.686647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec85d7d-13a8-42cb-9ad5-ec91da12c897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:57.701035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25600e0-9e06-4015-a97c-85c3381bf25f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:57.704572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25600e0-9e06-4015-a97c-85c3381bf25f service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:57.704572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25600e0-9e06-4015-a97c-85c3381bf25f service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:57.704572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25600e0-9e06-4015-a97c-85c3381bf25f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:57.704572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d25600e0-9e06-4015-a97c-85c3381bf25f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:57.706740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9ec85d7d-13a8-42cb-9ad5-ec91da12c897 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:57.707926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9ec85d7d-13a8-42cb-9ad5-ec91da12c897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:57.756575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:57.772004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25600e0-9e06-4015-a97c-85c3381bf25f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:57.773639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25600e0-9e06-4015-a97c-85c3381bf25f service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:57.782226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:47:57.798169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 901/1821] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:47:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:47:57.804353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 921/1822] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:47:57.804652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d7173-9bc2-45fa-b9c3-fb9cd25a8ab9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:57.804978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d7173-9bc2-45fa-b9c3-fb9cd25a8ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:57.805974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d7173-9bc2-45fa-b9c3-fb9cd25a8ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:57.829275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1c4d7173-9bc2-45fa-b9c3-fb9cd25a8ab9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:57.830552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1c4d7173-9bc2-45fa-b9c3-fb9cd25a8ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:57.843958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec85d7d-13a8-42cb-9ad5-ec91da12c897 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:57.854597 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 902/1823] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:57.860229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec85d7d-13a8-42cb-9ad5-ec91da12c897 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:57.872661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:57.872661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:57.872661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:57.872661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:57.873069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:47:57.873069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:57.873069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:57.889377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d7173-9bc2-45fa-b9c3-fb9cd25a8ab9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:57.895521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 922/1824] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:47:57.905283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d7173-9bc2-45fa-b9c3-fb9cd25a8ab9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:57.925315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:57.928857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:57.928857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:57.928857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:57.928857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:57.929442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:58.045581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 903/1825] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:57 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:47:58.050068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:58.063003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fbc0b-6f63-4b3e-aba7-1c25f1252196 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:58.149554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:58.149554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:58.149554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:58.149554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:58.149926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:58.149926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:58.870813 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 923/1826] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:58 2022] POST /identity/v3/users => generated 293 bytes in 818 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:58.872686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:58.884065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063c20-af55-4cf0-bf8e-88aa2f91b8d7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:58.888627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:58.889316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:58.889606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:58.889888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:58.890619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:58.892240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:58.903834 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 904/1827] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:58.907855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:58.918705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2319d-a5cb-428d-9451-d369417a643f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:58.923467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0af5c-3153-4663-a8d0-e4c1da0d1d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:58.924140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0af5c-3153-4663-a8d0-e4c1da0d1d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:58.924427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0af5c-3153-4663-a8d0-e4c1da0d1d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:58.924753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0af5c-3153-4663-a8d0-e4c1da0d1d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:58.925492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0af5c-3153-4663-a8d0-e4c1da0d1d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:58.927100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0af5c-3153-4663-a8d0-e4c1da0d1d24 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:58.938274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 924/1828] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:47:58.942598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:58.953367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c3504-8a3d-4164-ad17-49fe7ab06d57 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:58.958048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:58.958945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:58.959238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:58.959525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] PATH_INFO: `/v3/projects/6e4ff96e85c44ddd91a98773750a48b0/users/5ad0cf28ff7047969a47cd07f23fb081/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:58.974233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4ff96e85c44ddd91a98773750a48b0, user_id=5ad0cf28ff7047969a47cd07f23fb081, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:58.977022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:59.400497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 905/1829] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:47:58 2022] PUT /identity/v3/projects/6e4ff96e85c44ddd91a98773750a48b0/users/5ad0cf28ff7047969a47cd07f23fb081/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:47:59.411711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224fd86-6147-4f79-9481-f4c30480b8a0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:59.412005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224fd86-6147-4f79-9481-f4c30480b8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:59.412293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224fd86-6147-4f79-9481-f4c30480b8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:59.444881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3224fd86-6147-4f79-9481-f4c30480b8a0 None None] MFA Rules not processed for user `5ad0cf28ff7047969a47cd07f23fb081`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:47:59.446234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3224fd86-6147-4f79-9481-f4c30480b8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:47:59.516856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 925/1830] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:47:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:47:59.521021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:59.620408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8feca94-0cf8-4635-81ce-12b2219e5c36 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:59.637170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:59.638641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:59.638641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:59.638641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:59.639153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:59.641937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:47:59.799519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 906/1831] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:47:59 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:47:59.801578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224fd86-6147-4f79-9481-f4c30480b8a0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:47:59.814598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224fd86-6147-4f79-9481-f4c30480b8a0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:47:59.885343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:47:59.886138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:47:59.886447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:47:59.886809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:47:59.887430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:47:59.889916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:00.037943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 926/1832] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:47:59 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:00.043706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:00.061759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33accbf-d243-4a77-8aba-190a87e77670 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:00.070257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:00.072516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:00.073298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:00.074358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:00.076865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:00.080086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:00.102250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 907/1833] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:00.109944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:00.134728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041ef9-df56-479c-aa25-f17c4e2bd182 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:00.142306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:00.143669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:00.144349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:00.145069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] PATH_INFO: `/v3/projects/4f594ea5a98e41799e6bae49205ec7e1/users/8d7601db2e704d2bb074b67f435fa91d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:00.161702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f594ea5a98e41799e6bae49205ec7e1, user_id=8d7601db2e704d2bb074b67f435fa91d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:00.164412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:00.261993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 927/1834] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:00 2022] PUT /identity/v3/projects/4f594ea5a98e41799e6bae49205ec7e1/users/8d7601db2e704d2bb074b67f435fa91d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:00.263258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:00.276038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbbe14-6140-47bc-8cf6-3e1a0b29d449 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:00.380230 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf8640-7773-4b80-a9a4-0a0674738b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:00.382552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf8640-7773-4b80-a9a4-0a0674738b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:00.382904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf8640-7773-4b80-a9a4-0a0674738b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:00.383242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf8640-7773-4b80-a9a4-0a0674738b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:00.384493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf8640-7773-4b80-a9a4-0a0674738b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:00.386208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf8640-7773-4b80-a9a4-0a0674738b9f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:00.422338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:00.423823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 908/1835] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:00.443547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df087ae-0e80-4984-9985-092475c61fe9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:00.457127 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:00.458359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:00.458805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:00.459250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] PATH_INFO: `/v3/projects/4f594ea5a98e41799e6bae49205ec7e1/users/8d7601db2e704d2bb074b67f435fa91d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:00.464037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f594ea5a98e41799e6bae49205ec7e1, user_id=8d7601db2e704d2bb074b67f435fa91d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:00.469431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:00.497387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 928/1836] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:00 2022] PUT /identity/v3/projects/4f594ea5a98e41799e6bae49205ec7e1/users/8d7601db2e704d2bb074b67f435fa91d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:00.500791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:00.501271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:00.501722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:00.535491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] MFA Rules not processed for user `8d7601db2e704d2bb074b67f435fa91d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:00.537063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:00.626501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 909/1837] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:00.641057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:00.641057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:00.718031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8cb27-4d85-4886-bc22-f1e0d769362a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:00.722115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b5ffb-9f2d-4a68-bfa0-997580864459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:00.723250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5ffb-9f2d-4a68-bfa0-997580864459 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:00.723250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5ffb-9f2d-4a68-bfa0-997580864459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:00.723436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5ffb-9f2d-4a68-bfa0-997580864459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:00.767956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:00.786908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b5ffb-9f2d-4a68-bfa0-997580864459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:00.788361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b5ffb-9f2d-4a68-bfa0-997580864459 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:00.798502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 929/1838] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:00.927058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:00.928588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:00.954494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f2fc8-a580-434b-8f0b-5903e229ea26 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:00.964209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cddb69-4577-4dd5-ad0b-cbce4534e97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:00.974990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cddb69-4577-4dd5-ad0b-cbce4534e97c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:00.974990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cddb69-4577-4dd5-ad0b-cbce4534e97c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:00.976776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cddb69-4577-4dd5-ad0b-cbce4534e97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:00.988134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-43cddb69-4577-4dd5-ad0b-cbce4534e97c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:01.071992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cddb69-4577-4dd5-ad0b-cbce4534e97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:01.074608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cddb69-4577-4dd5-ad0b-cbce4534e97c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:01.095830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 910/1839] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:02.088323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:03.094523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446586b-0a3a-4412-b009-0e534b14ecaf None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:03.095167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446586b-0a3a-4412-b009-0e534b14ecaf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:03.095419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446586b-0a3a-4412-b009-0e534b14ecaf None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:03.097011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 930/1840] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:48:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:48:03.107086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af3cc-9617-44ef-8e6c-882c2b99089e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:03.107530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af3cc-9617-44ef-8e6c-882c2b99089e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:03.107650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522af3cc-9617-44ef-8e6c-882c2b99089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:03.126421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-522af3cc-9617-44ef-8e6c-882c2b99089e None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:03.127630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-522af3cc-9617-44ef-8e6c-882c2b99089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:03.202128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 911/1841] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:48:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:03.221424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446586b-0a3a-4412-b009-0e534b14ecaf None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:03.226321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b446586b-0a3a-4412-b009-0e534b14ecaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:03.253161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446586b-0a3a-4412-b009-0e534b14ecaf None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:03.260299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:03.261030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:03.261342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:03.261646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:03.273274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:03.275510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:03.293586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 931/1842] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:04.182032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:04.324580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:07.497087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522af3cc-9617-44ef-8e6c-882c2b99089e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:07.534191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522af3cc-9617-44ef-8e6c-882c2b99089e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:07.537935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372280f6-20ea-4e5e-94c4-6880dbdef957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:07.538581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372280f6-20ea-4e5e-94c4-6880dbdef957 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:07.538866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372280f6-20ea-4e5e-94c4-6880dbdef957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:07.539139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372280f6-20ea-4e5e-94c4-6880dbdef957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:07.550725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372280f6-20ea-4e5e-94c4-6880dbdef957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:07.552075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372280f6-20ea-4e5e-94c4-6880dbdef957 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:07.563216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 912/1843] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:08.059221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:08.237181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:10.610572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:20.657124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:21.681495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=20 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:22.070222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:22.133399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf819dce-770d-4365-96b6-bc11eca45beb service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:22.137896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:22.138754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:22.139034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:22.139308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] PATH_INFO: `/v3/users/5ad0cf28ff7047969a47cd07f23fb081` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:22.140486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad0cf28ff7047969a47cd07f23fb081)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:22.142537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:22.352823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad0cf28ff7047969a47cd07f23fb081'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:22.364920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad0cf28ff7047969a47cd07f23fb081'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:22.365740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad0cf28ff7047969a47cd07f23fb081'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:22.372861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad0cf28ff7047969a47cd07f23fb081'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:22.373525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad0cf28ff7047969a47cd07f23fb081'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:22.498557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 932/1844] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:22 2022] DELETE /identity/v3/users/5ad0cf28ff7047969a47cd07f23fb081 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:22.503249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:22.503411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:22.503663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:22.523168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:22.524422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:22.586242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 913/1845] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:22.596545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:22.598201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:22.669676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9e64e-abcd-4ee5-b523-875d8d60e1d2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:22.674248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:22.674913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:22.675200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:22.675478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:22.685605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:22.687570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:22.697699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 933/1846] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:23.257924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:23.259361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:23.272947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edb78e-ef81-461a-aaf9-fd60a22400f9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:23.277833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:23.281147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:23.281147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:23.281147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] PATH_INFO: `/v3/projects/6e4ff96e85c44ddd91a98773750a48b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:23.281416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4ff96e85c44ddd91a98773750a48b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:23.283126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:23.440761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4ff96e85c44ddd91a98773750a48b0', 'request_id': 'req-fc09c907-36a0-44f2-bf18-982b49871dbe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:23.447100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4ff96e85c44ddd91a98773750a48b0', 'request_id': 'req-fc09c907-36a0-44f2-bf18-982b49871dbe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:23.547289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4ff96e85c44ddd91a98773750a48b0', 'request_id': 'req-fc09c907-36a0-44f2-bf18-982b49871dbe'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:23.558548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] The token cache is being invalidate because project 6e4ff96e85c44ddd91a98773750a48b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:23.558836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:23.559462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:23.561064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 914/1847] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:23 2022] DELETE /identity/v3/projects/6e4ff96e85c44ddd91a98773750a48b0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:23.565522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:23.577733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae01883-b478-487d-b5a9-7e1d2c5152a9 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:23.597904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:23.636708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:23.637811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:23.638101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:23.638379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] PATH_INFO: `/v3/users/8d7601db2e704d2bb074b67f435fa91d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:23.640139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7601db2e704d2bb074b67f435fa91d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:23.642149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:24.022666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7601db2e704d2bb074b67f435fa91d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:24.034724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7601db2e704d2bb074b67f435fa91d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:24.035339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7601db2e704d2bb074b67f435fa91d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:24.042373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7601db2e704d2bb074b67f435fa91d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:24.042992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7601db2e704d2bb074b67f435fa91d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:24.239914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 934/1848] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:23 2022] DELETE /identity/v3/users/8d7601db2e704d2bb074b67f435fa91d => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:24.702492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:24.705149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:24.732516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09c907-36a0-44f2-bf18-982b49871dbe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:24.793828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:24.794710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:24.794995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:24.795274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] PATH_INFO: `/v3/users/48eeb6c70cf8439c9a23d9f3d2e4a3ea` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:24.797071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48eeb6c70cf8439c9a23d9f3d2e4a3ea)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:24.799005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:24.918705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:24.959806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:24.962464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:24.979458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694db8-c4f1-4ec3-b922-a306133b8731 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:24.985056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:24.985899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:24.986080 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:24.986347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] PATH_INFO: `/v3/projects/4f594ea5a98e41799e6bae49205ec7e1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:24.987902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f594ea5a98e41799e6bae49205ec7e1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:24.989886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.007752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48eeb6c70cf8439c9a23d9f3d2e4a3ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.017268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48eeb6c70cf8439c9a23d9f3d2e4a3ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.134924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48eeb6c70cf8439c9a23d9f3d2e4a3ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.135824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48eeb6c70cf8439c9a23d9f3d2e4a3ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.139051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-00f885f5-ec16-4b1a-8c16-000773ef47d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48eeb6c70cf8439c9a23d9f3d2e4a3ea'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.165512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 915/1849] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:24 2022] DELETE /identity/v3/users/48eeb6c70cf8439c9a23d9f3d2e4a3ea => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:25.167944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f594ea5a98e41799e6bae49205ec7e1', 'request_id': 'req-f01a6eec-0903-40b3-922f-d8b0de644ed3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.182560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f594ea5a98e41799e6bae49205ec7e1', 'request_id': 'req-f01a6eec-0903-40b3-922f-d8b0de644ed3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.192290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f594ea5a98e41799e6bae49205ec7e1', 'request_id': 'req-f01a6eec-0903-40b3-922f-d8b0de644ed3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.300377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] The token cache is being invalidate because project 4f594ea5a98e41799e6bae49205ec7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:25.300738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.301408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f01a6eec-0903-40b3-922f-d8b0de644ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.302996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 935/1850] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:24 2022] DELETE /identity/v3/projects/4f594ea5a98e41799e6bae49205ec7e1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:25.308976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751cd1-5d22-4255-8d8c-8ecd2e2a3d2e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.309353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751cd1-5d22-4255-8d8c-8ecd2e2a3d2e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.309709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751cd1-5d22-4255-8d8c-8ecd2e2a3d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.329401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f2751cd1-5d22-4255-8d8c-8ecd2e2a3d2e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:25.332497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f2751cd1-5d22-4255-8d8c-8ecd2e2a3d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:25.392386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 916/1851] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:25.400815 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.401112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.401542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.420302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:25.422005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:25.429949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 936/1852] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:25.434189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2751cd1-5d22-4255-8d8c-8ecd2e2a3d2e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.444360 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2751cd1-5d22-4255-8d8c-8ecd2e2a3d2e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.448876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.449241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.450103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.450544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.450774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:25.451278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.452197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:25.452709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.454579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.463354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f92d3-2971-49dd-b6b9-3a5836e52d7a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.474275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 917/1853] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:48:25.474483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.474483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.474483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.474483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] PATH_INFO: `/v3/projects/a8064e0a94be4b918b133154eb1aefa4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.475887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8064e0a94be4b918b133154eb1aefa4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.476005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.478224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.486940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6affd-bfa4-4e8c-8523-95fccc1e2f8c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.491934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.492662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.492959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.493090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:25.493241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.494102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.496081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.600444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 918/1854] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:25 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:25.601516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.608224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8064e0a94be4b918b133154eb1aefa4', 'request_id': 'req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.614921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8064e0a94be4b918b133154eb1aefa4', 'request_id': 'req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.620464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8064e0a94be4b918b133154eb1aefa4', 'request_id': 'req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.661337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a236f9-fbbb-47df-8c8f-9b82ddea858e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.665928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.666598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.666883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.667162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.667853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.669811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.694238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] The token cache is being invalidate because project a8064e0a94be4b918b133154eb1aefa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:25.694424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.695156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.696787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 937/1855] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:25 2022] DELETE /identity/v3/projects/a8064e0a94be4b918b133154eb1aefa4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:25.700650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.702656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:25.766739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fca99-9407-4854-995f-b9f3c1e5bb1a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.771397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.772284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.772572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.772909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] PATH_INFO: `/v3/users/2a607ebdf7354405bedc9f546416cb39` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.774519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a607ebdf7354405bedc9f546416cb39)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.776499 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.781690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 919/1856] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:25 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:25.785886 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.787586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:25.799841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e942c-a657-41b2-8ab4-c92bf2bc963c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.804420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.805124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.805413 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.805688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.806406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.808010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.818333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 920/1857] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:25.822259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.832161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ca002-5d50-475e-ac10-a5b116c77b0b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.836886 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.837522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.837808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.838094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.838782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.840357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.844435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:25.853011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 921/1858] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:25.856865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:25.866737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b77da6-21c5-43f5-ab96-9c20b04aeebf admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:25.871274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b46e7-d66c-4255-8360-9ca2f5e574f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:25.872118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b46e7-d66c-4255-8360-9ca2f5e574f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:25.872402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b46e7-d66c-4255-8360-9ca2f5e574f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:25.872707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b46e7-d66c-4255-8360-9ca2f5e574f5 admin admin] PATH_INFO: `/v3/projects/e630ee805634497882a66805f99e5e6d/users/dbe8227bff394ab384d640bbbacdd510/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:25.887209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b46e7-d66c-4255-8360-9ca2f5e574f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e630ee805634497882a66805f99e5e6d, user_id=dbe8227bff394ab384d640bbbacdd510, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:25.889977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b46e7-d66c-4255-8360-9ca2f5e574f5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:25.987045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a607ebdf7354405bedc9f546416cb39'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:25.991753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 922/1859] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:25 2022] PUT /identity/v3/projects/e630ee805634497882a66805f99e5e6d/users/dbe8227bff394ab384d640bbbacdd510/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:26.003048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e501-69d4-47a0-a0d1-ce58cfae1882 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.003341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e501-69d4-47a0-a0d1-ce58cfae1882 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.003623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e501-69d4-47a0-a0d1-ce58cfae1882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.004562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a607ebdf7354405bedc9f546416cb39'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.005585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a607ebdf7354405bedc9f546416cb39'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.012913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a607ebdf7354405bedc9f546416cb39'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.013617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a607ebdf7354405bedc9f546416cb39'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.031067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8db1e501-69d4-47a0-a0d1-ce58cfae1882 None None] MFA Rules not processed for user `dbe8227bff394ab384d640bbbacdd510`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:26.032270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8db1e501-69d4-47a0-a0d1-ce58cfae1882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:26.088782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 938/1860] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:25 2022] DELETE /identity/v3/users/2a607ebdf7354405bedc9f546416cb39 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:26.094824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 923/1861] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:26.099173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.100556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:26.174329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0214b-fd56-43fd-8af2-4ed23f3eadca admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.179890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.180570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.181509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.181796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.182520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.185100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.226749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:26.299039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 939/1862] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:26 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:26.303136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1e501-69d4-47a0-a0d1-ce58cfae1882 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.368351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1e501-69d4-47a0-a0d1-ce58cfae1882 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.383920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.386393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:26.441537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.442248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.442532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.442808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.443513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.445481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.474612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed525ab-54b0-4eb3-aa84-873097a9b3d0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.479316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.480183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.480472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.480787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] PATH_INFO: `/v3/projects/5bac4a8fd2a1431892c25a6467d00ffd` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.482013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bac4a8fd2a1431892c25a6467d00ffd)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.483960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.597788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 924/1863] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:26 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:26.601715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.612698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44a5b4-ae46-494c-adc3-b8b0b9b77060 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.619926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.619926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.619926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.619926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.620282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.621338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.637257 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 925/1864] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:26.638524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.643359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bac4a8fd2a1431892c25a6467d00ffd', 'request_id': 'req-019f43d7-f6b0-4313-a299-02cb9dd18d40'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.649241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9a21-53ff-4666-8768-ba1ed986befd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.649823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bac4a8fd2a1431892c25a6467d00ffd', 'request_id': 'req-019f43d7-f6b0-4313-a299-02cb9dd18d40'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.655508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bac4a8fd2a1431892c25a6467d00ffd', 'request_id': 'req-019f43d7-f6b0-4313-a299-02cb9dd18d40'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.713316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.714212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.714502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.714803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] PATH_INFO: `/v3/projects/15fccfce05bb4feb853e086a154e9deb/users/1a02b527f18945f091f5bddd67af9ef7/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.729273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=15fccfce05bb4feb853e086a154e9deb, user_id=1a02b527f18945f091f5bddd67af9ef7, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.731987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.734843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] The token cache is being invalidate because project 5bac4a8fd2a1431892c25a6467d00ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:26.735132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.735799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-019f43d7-f6b0-4313-a299-02cb9dd18d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:26.737446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 940/1865] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:26 2022] DELETE /identity/v3/projects/5bac4a8fd2a1431892c25a6467d00ffd => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:26.742149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b49c-77da-4e03-821a-394351239a4e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.742433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b49c-77da-4e03-821a-394351239a4e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.742712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b49c-77da-4e03-821a-394351239a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.763247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a6b7b49c-77da-4e03-821a-394351239a4e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:26.765388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a6b7b49c-77da-4e03-821a-394351239a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:26.773615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 941/1866] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:26.782188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ca26-3910-46e3-99a6-abe0277dc457 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.782476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ca26-3910-46e3-99a6-abe0277dc457 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.782752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7ca26-3910-46e3-99a6-abe0277dc457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.792045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:26.800497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d2f7ca26-3910-46e3-99a6-abe0277dc457 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:26.801720 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d2f7ca26-3910-46e3-99a6-abe0277dc457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:26.809656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 942/1867] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:26.813841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7ca26-3910-46e3-99a6-abe0277dc457 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.822826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7ca26-3910-46e3-99a6-abe0277dc457 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.828071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.828728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.829013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.829289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.830000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:26.830303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.831881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.833016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 926/1868] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:26 2022] PUT /identity/v3/projects/15fccfce05bb4feb853e086a154e9deb/users/1a02b527f18945f091f5bddd67af9ef7/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:26.841089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.843341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:26.844741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 943/1869] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:48:26.848920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.862058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1520af5-9966-4a83-8f2c-faead91844e9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.914919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffdd42-5824-4980-8649-bfca8bc62bfe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.917892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.918558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.918840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.919110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.919803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.921971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.922115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.922663 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.927856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.927856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.927856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.927856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.936364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 927/1870] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:26.940861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:26.950521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ddbf7-d1c7-45b3-9ad9-2c5ef75f874e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:26.955991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3ed83-3e9e-416f-9b58-9548cc20fbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:26.958881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3ed83-3e9e-416f-9b58-9548cc20fbb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.958881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3ed83-3e9e-416f-9b58-9548cc20fbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.958881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3ed83-3e9e-416f-9b58-9548cc20fbb8 admin admin] PATH_INFO: `/v3/projects/15fccfce05bb4feb853e086a154e9deb/users/1a02b527f18945f091f5bddd67af9ef7/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:26.959407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3ed83-3e9e-416f-9b58-9548cc20fbb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15fccfce05bb4feb853e086a154e9deb, user_id=1a02b527f18945f091f5bddd67af9ef7, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:26.962118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3ed83-3e9e-416f-9b58-9548cc20fbb8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:26.972766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 928/1871] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:26 2022] PUT /identity/v3/projects/15fccfce05bb4feb853e086a154e9deb/users/1a02b527f18945f091f5bddd67af9ef7/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:26.984850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:26.985423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:26.985581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.016106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] MFA Rules not processed for user `1a02b527f18945f091f5bddd67af9ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:27.017366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:27.041337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 944/1872] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:26 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:27.046396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:27.091467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 929/1873] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:27.103179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:27.104480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:27.113242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d268e-ed60-4d8d-9217-9fabcdcf01ae admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:27.117949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:27.118620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.118906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.119180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.119873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:27.121849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:27.162068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d6701-b89f-40c2-bdac-6f8ba61e0ada None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:27.209873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:27.210534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.210823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.211102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.212928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:27.237902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:27.274779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:27.276232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:27.285095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 930/1874] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:48:27.295380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 945/1875] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:27 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:27.305705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:27.317415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca42b9-2e5b-426e-b91d-f61087d559d7 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:27.322306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:27.322977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.323267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.323550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.324282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:27.325919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:27.338474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 931/1876] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:27.343001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:27.362507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36282079-eba6-49bb-9141-8f6d657cfee6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:27.367398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfd60b-dad2-4c58-bbd0-7abef35e1535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:27.368076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfd60b-dad2-4c58-bbd0-7abef35e1535 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.368390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfd60b-dad2-4c58-bbd0-7abef35e1535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.368705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfd60b-dad2-4c58-bbd0-7abef35e1535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.369446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfd60b-dad2-4c58-bbd0-7abef35e1535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:27.371090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfd60b-dad2-4c58-bbd0-7abef35e1535 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:27.382944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 946/1877] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:27.387614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:27.398305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:27.406926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dcd5e-8da4-4172-b2ce-81673e20bf1d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:27.413342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:27.414240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.414531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.414810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] PATH_INFO: `/v3/projects/b93b76663bf94102aea554c5e8574ed4/users/d39c112f3a5e4dab94ff0320bb04cfab/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.432053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93b76663bf94102aea554c5e8574ed4, user_id=d39c112f3a5e4dab94ff0320bb04cfab, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:27.435443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:27.526441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 932/1878] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:27 2022] PUT /identity/v3/projects/b93b76663bf94102aea554c5e8574ed4/users/d39c112f3a5e4dab94ff0320bb04cfab/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:27.537258 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d8366-c5f0-4ae7-9625-77ad3cac6d15 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.537553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d8366-c5f0-4ae7-9625-77ad3cac6d15 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.537833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d8366-c5f0-4ae7-9625-77ad3cac6d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.565130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-163d8366-c5f0-4ae7-9625-77ad3cac6d15 None None] MFA Rules not processed for user `d39c112f3a5e4dab94ff0320bb04cfab`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:27.566338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-163d8366-c5f0-4ae7-9625-77ad3cac6d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:27.631705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 947/1879] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:27.643293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:27.645041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:27.648741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:27.704330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59686996-a347-4381-b32b-bafd927a767e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:27.708263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea23b2-42dc-41b6-8048-022e248111ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:27.711644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea23b2-42dc-41b6-8048-022e248111ce service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.711644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea23b2-42dc-41b6-8048-022e248111ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.711644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea23b2-42dc-41b6-8048-022e248111ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.720420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea23b2-42dc-41b6-8048-022e248111ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:27.721942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea23b2-42dc-41b6-8048-022e248111ce service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:27.731838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 933/1880] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:27.751528 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d8366-c5f0-4ae7-9625-77ad3cac6d15 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:27.756089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:27.815015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d8366-c5f0-4ae7-9625-77ad3cac6d15 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:27.819666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:27.820521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:27.820841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:27.821123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] PATH_INFO: `/v3/users/d39c112f3a5e4dab94ff0320bb04cfab` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:27.822333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39c112f3a5e4dab94ff0320bb04cfab)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:27.824300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:27.982943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39c112f3a5e4dab94ff0320bb04cfab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:27.995448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39c112f3a5e4dab94ff0320bb04cfab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:27.996343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39c112f3a5e4dab94ff0320bb04cfab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:28.006278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39c112f3a5e4dab94ff0320bb04cfab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:28.006642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39c112f3a5e4dab94ff0320bb04cfab'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:28.096897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 948/1881] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:27 2022] DELETE /identity/v3/users/d39c112f3a5e4dab94ff0320bb04cfab => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:28.101401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:28.103711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:28.103711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:28.122862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:28.124126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:28.186552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 934/1882] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:28.196513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:28.197977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:28.258589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377956a-7a42-42ef-b380-db0c41f0d30c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:28.262277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ebaf6-38d0-4dae-b92d-5b9826797939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:28.262938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebaf6-38d0-4dae-b92d-5b9826797939 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:28.263219 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebaf6-38d0-4dae-b92d-5b9826797939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:28.263498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ebaf6-38d0-4dae-b92d-5b9826797939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:28.274868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ebaf6-38d0-4dae-b92d-5b9826797939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:28.276331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ebaf6-38d0-4dae-b92d-5b9826797939 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:28.286787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 949/1883] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:28.867775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:28.869488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:28.883105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357f543-bddd-46b5-b9ad-e555989102ab None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:28.887757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:28.891691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:28.891691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:28.891691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] PATH_INFO: `/v3/projects/b93b76663bf94102aea554c5e8574ed4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:28.891691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93b76663bf94102aea554c5e8574ed4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:28.892983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:29.022580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93b76663bf94102aea554c5e8574ed4', 'request_id': 'req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:29.028873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93b76663bf94102aea554c5e8574ed4', 'request_id': 'req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:29.131316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93b76663bf94102aea554c5e8574ed4', 'request_id': 'req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:29.134175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] The token cache is being invalidate because project b93b76663bf94102aea554c5e8574ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:29.134577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:29.135122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b7e7f49a-230f-43d8-a1b7-9d7385ff8cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:29.136688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 935/1884] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:28 2022] DELETE /identity/v3/projects/b93b76663bf94102aea554c5e8574ed4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:29.142113 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2db27a-ff18-473b-b2a7-316da664e3ac None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.142690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2db27a-ff18-473b-b2a7-316da664e3ac None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.142690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2db27a-ff18-473b-b2a7-316da664e3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.161057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cf2db27a-ff18-473b-b2a7-316da664e3ac None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:29.162275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cf2db27a-ff18-473b-b2a7-316da664e3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:29.181779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:29.222143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 950/1885] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:29.229922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba25309-ca41-439e-8a16-dacaacfce5d9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.230217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba25309-ca41-439e-8a16-dacaacfce5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.230598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba25309-ca41-439e-8a16-dacaacfce5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.253635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-dba25309-ca41-439e-8a16-dacaacfce5d9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:29.254837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dba25309-ca41-439e-8a16-dacaacfce5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:29.263072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 936/1886] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:29.267542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2db27a-ff18-473b-b2a7-316da664e3ac None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:29.277507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2db27a-ff18-473b-b2a7-316da664e3ac None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:29.282254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:29.283130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.283429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.283887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.285020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:29.285373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:29.286993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:29.300067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 951/1887] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:48:29.304367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba25309-ca41-439e-8a16-dacaacfce5d9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:29.315176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba25309-ca41-439e-8a16-dacaacfce5d9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:29.320348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:29.321179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.321548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.322015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.323018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:29.325181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:29.338982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:29.469316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 937/1888] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:29 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:29.474608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:29.485630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846885c-851b-4ed5-8b2c-84b41ae9e063 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:29.548096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:29.548845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.549631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.549918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.550675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:29.552630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:29.709974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 952/1889] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:29 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:29.712285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:29.724003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc477106-7dba-4579-be4e-96c1f3eca2fc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:29.729677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:29.729677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.729677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.729961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.730566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:29.732202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:29.743229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 938/1890] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:29.747474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:29.758647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ea5bd-2e34-4756-b57a-fb04b6324e4b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:29.763263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24983-247c-4801-8fa8-b814683eab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:29.763928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24983-247c-4801-8fa8-b814683eab5b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.764267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24983-247c-4801-8fa8-b814683eab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.764547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24983-247c-4801-8fa8-b814683eab5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.765320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24983-247c-4801-8fa8-b814683eab5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:29.766915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24983-247c-4801-8fa8-b814683eab5b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:29.777131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 953/1891] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:29.781996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:29.792801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9f71c-6188-4802-a67c-d69830847b8a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:29.798611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:29.799610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.799974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.801603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] PATH_INFO: `/v3/projects/a65b261bba734803b7d29d5bf0bf5abe/users/c65f6f766b594d8b823ecbb7b5fe35e5/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.816455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65b261bba734803b7d29d5bf0bf5abe, user_id=c65f6f766b594d8b823ecbb7b5fe35e5, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:29.819245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:29.916453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 939/1892] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:29 2022] PUT /identity/v3/projects/a65b261bba734803b7d29d5bf0bf5abe/users/c65f6f766b594d8b823ecbb7b5fe35e5/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:29.929708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16721-5d1e-480e-b931-fa50eac7dd65 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:29.929708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16721-5d1e-480e-b931-fa50eac7dd65 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:29.929708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16721-5d1e-480e-b931-fa50eac7dd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:29.964059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-19d16721-5d1e-480e-b931-fa50eac7dd65 None None] MFA Rules not processed for user `c65f6f766b594d8b823ecbb7b5fe35e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:29.965436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-19d16721-5d1e-480e-b931-fa50eac7dd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:30.036440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 954/1893] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:30.041023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:30.104410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e96ddb-1778-45e9-b3d8-2e45ab96d341 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:30.109173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:30.109832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.110123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.110404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.111095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:30.113052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:30.205326 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 940/1894] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:30 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:30.209408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16721-5d1e-480e-b931-fa50eac7dd65 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:30.220629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16721-5d1e-480e-b931-fa50eac7dd65 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:30.238256 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:30.281157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:30.281830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.282969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.282969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.283189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:30.285187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:30.385233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 955/1895] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:30 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:30.389266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:30.399681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8678-438d-4f0b-8721-ff0e7230e840 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:30.404428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:30.405639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.405924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.406203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.406930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:30.408544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:30.419069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 941/1896] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:30.423723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:30.433916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80a878-8ee9-417c-88e9-d55933b494de admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:30.439326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86faa0-f94e-4e18-80af-f90ef551e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:30.440310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86faa0-f94e-4e18-80af-f90ef551e8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.440310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86faa0-f94e-4e18-80af-f90ef551e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.440526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86faa0-f94e-4e18-80af-f90ef551e8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.441298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86faa0-f94e-4e18-80af-f90ef551e8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:30.442903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86faa0-f94e-4e18-80af-f90ef551e8e5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:30.454835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 956/1897] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:30.458937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:30.469218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acb82b-bf18-4b56-95f7-2756666f6e2d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:30.473919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:30.474983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.475232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.475541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] PATH_INFO: `/v3/projects/4888b27bb0294677a0bc6c08ff5dc7c5/users/04db0de1ac4d48efac39be083692a187/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.491684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4888b27bb0294677a0bc6c08ff5dc7c5, user_id=04db0de1ac4d48efac39be083692a187, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:30.494425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:30.580436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 942/1898] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:30 2022] PUT /identity/v3/projects/4888b27bb0294677a0bc6c08ff5dc7c5/users/04db0de1ac4d48efac39be083692a187/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:30.591783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2d182-d492-45eb-9c8e-b10417f780a4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.592097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2d182-d492-45eb-9c8e-b10417f780a4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.592455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2d182-d492-45eb-9c8e-b10417f780a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.619746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e6e2d182-d492-45eb-9c8e-b10417f780a4 None None] MFA Rules not processed for user `04db0de1ac4d48efac39be083692a187`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:30.621006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e6e2d182-d492-45eb-9c8e-b10417f780a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:30.684670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 957/1899] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:30.689635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:30.760929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c27c19-8ccb-49ba-b48c-9fb7915d9001 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:30.766366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:30.767017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.767304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.767580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.768278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:30.770253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:30.779707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:30.901294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 943/1900] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:30 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:30.907359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2d182-d492-45eb-9c8e-b10417f780a4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:30.918332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2d182-d492-45eb-9c8e-b10417f780a4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:30.976043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:30.976752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:30.977045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:30.977326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:30.978068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:30.979996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:31.087364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 958/1901] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:30 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:31.098582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:31.114598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ac67d-f62b-45ca-acc0-f955b639e04c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:31.119414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:31.120241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:31.120529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:31.120856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:31.121589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:31.123260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:31.134966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 944/1902] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:31.138878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:31.149284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcdb7-32d8-459b-b203-5f804e725ead admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:31.153838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:31.156342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:31.156342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:31.156342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] PATH_INFO: `/v3/projects/547f56e7145f44a3bce54a670191dbc2/users/095ca8c4075d40609fa4a8dc6f3efae3/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:31.174255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547f56e7145f44a3bce54a670191dbc2, user_id=095ca8c4075d40609fa4a8dc6f3efae3, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:31.177072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:31.179549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:31.284107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 959/1903] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:31 2022] PUT /identity/v3/projects/547f56e7145f44a3bce54a670191dbc2/users/095ca8c4075d40609fa4a8dc6f3efae3/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:31.288084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:31.298295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc908082-e328-45e8-a30b-21c8f55b67ea admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:31.352982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bc603-07be-41fb-b0b2-023bf7932cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:31.353638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bc603-07be-41fb-b0b2-023bf7932cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:31.353922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bc603-07be-41fb-b0b2-023bf7932cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:31.354201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bc603-07be-41fb-b0b2-023bf7932cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:31.354918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bc603-07be-41fb-b0b2-023bf7932cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:31.356545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bc603-07be-41fb-b0b2-023bf7932cf8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:31.366251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 945/1904] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:31.370419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:31.380999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7498a18-61e0-4374-8c3f-c919f7b2e968 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:31.385531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:31.386405 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:31.386686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:31.391297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] PATH_INFO: `/v3/projects/547f56e7145f44a3bce54a670191dbc2/users/095ca8c4075d40609fa4a8dc6f3efae3/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:31.391297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547f56e7145f44a3bce54a670191dbc2, user_id=095ca8c4075d40609fa4a8dc6f3efae3, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:31.392254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:31.402518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 960/1905] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:31 2022] PUT /identity/v3/projects/547f56e7145f44a3bce54a670191dbc2/users/095ca8c4075d40609fa4a8dc6f3efae3/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:31.413719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:31.414019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:31.414309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:31.439724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] MFA Rules not processed for user `095ca8c4075d40609fa4a8dc6f3efae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:31.441043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:31.511801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 946/1906] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:31.523744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:31.525524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:31.584230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f96b9-7e26-4754-bc68-6c9cfc518da2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:31.587913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:31.588562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:31.588883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:31.589166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:31.646207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:31.647634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:31.656551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 961/1907] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:33.620419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:33.621882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:33.696921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1413d36-dd51-4f8b-9114-d2f1ef1aca0a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:33.702421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:33.703368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:33.703776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:33.703924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] PATH_INFO: `/v3/users/dbe8227bff394ab384d640bbbacdd510` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:33.705216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbe8227bff394ab384d640bbbacdd510)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:33.707416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:33.710561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:33.821488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:33.830323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d526e-9b97-414b-93bb-33545e62f2ef service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:33.833972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357efbfe-d26c-4d1a-a125-3dde53441196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:33.834635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357efbfe-d26c-4d1a-a125-3dde53441196 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:33.834923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357efbfe-d26c-4d1a-a125-3dde53441196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:33.835206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357efbfe-d26c-4d1a-a125-3dde53441196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:33.891725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357efbfe-d26c-4d1a-a125-3dde53441196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:33.893205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357efbfe-d26c-4d1a-a125-3dde53441196 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:33.902271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 962/1908] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:48:33.953587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbe8227bff394ab384d640bbbacdd510'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:33.961194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbe8227bff394ab384d640bbbacdd510'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:34.071632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbe8227bff394ab384d640bbbacdd510'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:34.072272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbe8227bff394ab384d640bbbacdd510'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:34.073003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbe8227bff394ab384d640bbbacdd510'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:34.092289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 947/1909] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:33 2022] DELETE /identity/v3/users/dbe8227bff394ab384d640bbbacdd510 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:34.097045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:34.097342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:34.097646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:34.114766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:34.115979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:34.146039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:34.175976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 963/1910] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:34.185564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:34.187051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:34.247563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c9d38-cdba-4622-ae11-4c7995f58f93 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:34.251398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:34.252048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:34.252364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:34.252672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:34.262180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:34.263610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:34.273687 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 948/1911] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:35.025676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:35.029831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:35.053876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca0940-9e19-4252-b4af-b7f774277593 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:35.058852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:35.060035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:35.060409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:35.060785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] PATH_INFO: `/v3/projects/e630ee805634497882a66805f99e5e6d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:35.066498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e630ee805634497882a66805f99e5e6d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:35.066498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:35.252314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e630ee805634497882a66805f99e5e6d', 'request_id': 'req-227db9d6-0b75-405a-96a7-77e292e8ad22'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.259000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e630ee805634497882a66805f99e5e6d', 'request_id': 'req-227db9d6-0b75-405a-96a7-77e292e8ad22'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.265134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e630ee805634497882a66805f99e5e6d', 'request_id': 'req-227db9d6-0b75-405a-96a7-77e292e8ad22'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.370519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] The token cache is being invalidate because project e630ee805634497882a66805f99e5e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:35.371751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.372662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.375426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 964/1912] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:35 2022] DELETE /identity/v3/projects/e630ee805634497882a66805f99e5e6d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:35.379899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:35.393519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698848-caa5-47cc-aea0-836f478661e3 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:35.470087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:35.471728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:35.472170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:35.472464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] PATH_INFO: `/v3/users/1a02b527f18945f091f5bddd67af9ef7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:35.473954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a02b527f18945f091f5bddd67af9ef7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:35.476409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:35.615772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:35.756293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a02b527f18945f091f5bddd67af9ef7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.763997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a02b527f18945f091f5bddd67af9ef7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.909670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a02b527f18945f091f5bddd67af9ef7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.910302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a02b527f18945f091f5bddd67af9ef7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.910922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a02b527f18945f091f5bddd67af9ef7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:35.929618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 949/1913] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:35 2022] DELETE /identity/v3/users/1a02b527f18945f091f5bddd67af9ef7 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:35.950311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:35.951870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:36.024887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227db9d6-0b75-405a-96a7-77e292e8ad22 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:36.080971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca93b39-557a-42d3-9daf-7c62a1422859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:36.081657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca93b39-557a-42d3-9daf-7c62a1422859 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:36.081919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca93b39-557a-42d3-9daf-7c62a1422859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:36.082271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca93b39-557a-42d3-9daf-7c62a1422859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:36.084449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7ca93b39-557a-42d3-9daf-7c62a1422859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:36.115128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:36.160278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca93b39-557a-42d3-9daf-7c62a1422859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:36.160278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca93b39-557a-42d3-9daf-7c62a1422859 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:36.162108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 965/1914] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:36.440104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:36.441650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:36.523922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1b5ed-34bf-451f-b52f-2d8eb16535ca admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:36.529027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:36.529968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:36.530443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:36.530736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] PATH_INFO: `/v3/projects/15fccfce05bb4feb853e086a154e9deb` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:36.532171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15fccfce05bb4feb853e086a154e9deb)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:36.534875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:36.701776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15fccfce05bb4feb853e086a154e9deb', 'request_id': 'req-412b1989-4514-4f7b-a31f-03c664b01ac1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:36.707949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15fccfce05bb4feb853e086a154e9deb', 'request_id': 'req-412b1989-4514-4f7b-a31f-03c664b01ac1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:36.803132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15fccfce05bb4feb853e086a154e9deb', 'request_id': 'req-412b1989-4514-4f7b-a31f-03c664b01ac1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:36.817289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] The token cache is being invalidate because project 15fccfce05bb4feb853e086a154e9deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:36.820715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:36.820715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-412b1989-4514-4f7b-a31f-03c664b01ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:36.820715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 950/1915] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:36 2022] DELETE /identity/v3/projects/15fccfce05bb4feb853e086a154e9deb => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:36.826838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb522b-5a44-430e-9fee-d8d4827038dd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:36.827025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb522b-5a44-430e-9fee-d8d4827038dd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:36.828002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb522b-5a44-430e-9fee-d8d4827038dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:36.850957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-77cb522b-5a44-430e-9fee-d8d4827038dd None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:36.852556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-77cb522b-5a44-430e-9fee-d8d4827038dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:36.926914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 966/1916] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:36.936502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7176d-c262-4fb1-96ba-0253cb2f6655 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:36.936729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7176d-c262-4fb1-96ba-0253cb2f6655 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:36.937016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7176d-c262-4fb1-96ba-0253cb2f6655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:36.956752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c7d7176d-c262-4fb1-96ba-0253cb2f6655 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:36.958000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c7d7176d-c262-4fb1-96ba-0253cb2f6655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:36.966224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 951/1917] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:36.970541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb522b-5a44-430e-9fee-d8d4827038dd None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:36.980807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb522b-5a44-430e-9fee-d8d4827038dd None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:36.985895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:36.986736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:36.987029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:36.987317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:36.988145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:36.988540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:36.990365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:37.006324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 967/1918] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:48:37.010823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7176d-c262-4fb1-96ba-0253cb2f6655 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:37.021679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7176d-c262-4fb1-96ba-0253cb2f6655 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:37.026585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:37.027287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:37.027578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:37.027864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:37.028573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:37.030698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:37.163043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 952/1919] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:37 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:37.167323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:37.180703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654972d6-3231-4e4b-9c2a-3e92d9c1fbae admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:37.256659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:37.257339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:37.257712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:37.257990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:37.258689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:37.260642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:37.404044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 968/1920] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:37 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:37.408040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:37.418101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:37.418396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6415f9d-6482-4897-a23b-94712c4ab60c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:37.423405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:37.424063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:37.424347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:37.424653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:37.425380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:37.427004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:37.437490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 953/1921] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:37.441954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:37.453356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e24e-1c30-45bd-8e19-ceead84bcc6d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:37.458474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0959b9-7019-435e-bc09-83c28426d1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:37.459283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0959b9-7019-435e-bc09-83c28426d1ed admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:37.459470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0959b9-7019-435e-bc09-83c28426d1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:37.459752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0959b9-7019-435e-bc09-83c28426d1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:37.460645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0959b9-7019-435e-bc09-83c28426d1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:37.462260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0959b9-7019-435e-bc09-83c28426d1ed admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:37.477904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 969/1922] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:37.480718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:37.491094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fd960-d0a8-42ed-8546-58e7373e1af6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:37.497031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:37.498378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:37.498903 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:37.499292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] PATH_INFO: `/v3/projects/dca12b0e26544d01b85f5040be80f788/users/60a580a828fa46b99f3a251230ca4d6b/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:37.515682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca12b0e26544d01b85f5040be80f788, user_id=60a580a828fa46b99f3a251230ca4d6b, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:37.518571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:37.603655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 954/1923] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:37 2022] PUT /identity/v3/projects/dca12b0e26544d01b85f5040be80f788/users/60a580a828fa46b99f3a251230ca4d6b/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:37.615244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:37.615541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:37.616545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:37.646336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] MFA Rules not processed for user `60a580a828fa46b99f3a251230ca4d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:37.647558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:37.666198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:37.721965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 970/1924] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:37.880250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:37.883225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:37.945443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6490b-7e0f-4cd5-b981-5bb1d112ad9b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:37.949023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad6fea-f1e3-4d54-913c-a3b74a67668c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:37.949676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6fea-f1e3-4d54-913c-a3b74a67668c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:37.949965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6fea-f1e3-4d54-913c-a3b74a67668c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:37.950244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6fea-f1e3-4d54-913c-a3b74a67668c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:37.959225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad6fea-f1e3-4d54-913c-a3b74a67668c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:37.960702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad6fea-f1e3-4d54-913c-a3b74a67668c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:37.970305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 955/1925] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:38.182884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:38.184325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:38.278813 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2c8b2-9a05-417e-8257-29d37fc0b62c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:38.284962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:38.285847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:38.286204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:38.286554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] PATH_INFO: `/v3/users/b0fa7d2e028d49f9a2235c41fc1ffe77` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:38.287967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0fa7d2e028d49f9a2235c41fc1ffe77)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:38.290235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:38.564641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0fa7d2e028d49f9a2235c41fc1ffe77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:38.580412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0fa7d2e028d49f9a2235c41fc1ffe77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:38.581508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0fa7d2e028d49f9a2235c41fc1ffe77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:38.588919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0fa7d2e028d49f9a2235c41fc1ffe77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:38.589627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0fa7d2e028d49f9a2235c41fc1ffe77'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:38.687373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 971/1926] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:38 2022] DELETE /identity/v3/users/b0fa7d2e028d49f9a2235c41fc1ffe77 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:38.690511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:38.690993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:38.691096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:38.716192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:38.718434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:38.795443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 956/1927] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:38.805107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:38.806633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:38.854489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:38.870211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd23b2-b9d0-4a1c-8735-ad792a0fac61 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:38.875032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:38.875867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:38.876308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:38.876586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:38.886590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:38.888025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:38.897924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 972/1928] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:39.365073 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:39.366574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:39.380499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bede95-b5b1-4238-a42b-a68d7ced4856 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:39.385534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:39.386420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:39.386781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:39.386974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] PATH_INFO: `/v3/projects/234bd5a5ea18455eb476d5f01165b18f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:39.388182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=234bd5a5ea18455eb476d5f01165b18f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:39.390857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:39.543195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234bd5a5ea18455eb476d5f01165b18f', 'request_id': 'req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:39.550306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234bd5a5ea18455eb476d5f01165b18f', 'request_id': 'req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:39.637504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234bd5a5ea18455eb476d5f01165b18f', 'request_id': 'req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:39.644162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:39.650606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] The token cache is being invalidate because project 234bd5a5ea18455eb476d5f01165b18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:39.651076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:39.651877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:39.653591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 957/1929] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:39 2022] DELETE /identity/v3/projects/234bd5a5ea18455eb476d5f01165b18f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:39.658085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:39.676485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da847-86c7-48d3-9186-f3992d235f60 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:39.754010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:39.754952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:39.755489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:39.755768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] PATH_INFO: `/v3/users/1b3788f2a2844868a6607f452c84ee57` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:39.757116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3788f2a2844868a6607f452c84ee57)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:39.759101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:39.978124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3788f2a2844868a6607f452c84ee57'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:39.991660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3788f2a2844868a6607f452c84ee57'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:39.992478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3788f2a2844868a6607f452c84ee57'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.004284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3788f2a2844868a6607f452c84ee57'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.005240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9d8f8a25-2275-405e-b0af-07b73e7a67bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3788f2a2844868a6607f452c84ee57'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.110237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 973/1930] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:39 2022] DELETE /identity/v3/users/1b3788f2a2844868a6607f452c84ee57 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:40.572907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:40.575054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:40.592841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a3de6-5460-4cf9-b9bb-5d90e6d426fd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:40.661582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:40.662465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:40.662804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:40.663191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] PATH_INFO: `/v3/projects/5faf61c7105f462798841e6df7753ca7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:40.664482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5faf61c7105f462798841e6df7753ca7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:40.666451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:40.868298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5faf61c7105f462798841e6df7753ca7', 'request_id': 'req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.874799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5faf61c7105f462798841e6df7753ca7', 'request_id': 'req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.972322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5faf61c7105f462798841e6df7753ca7', 'request_id': 'req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.984296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] The token cache is being invalidate because project 5faf61c7105f462798841e6df7753ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:40.984582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.985248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-06ec9741-98cc-4dc0-bd28-3d59f5f9e5d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:40.986815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 958/1931] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:40 2022] DELETE /identity/v3/projects/5faf61c7105f462798841e6df7753ca7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:40.991936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f275b5-02bc-4d4c-97bd-da1343dfed43 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:40.992398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f275b5-02bc-4d4c-97bd-da1343dfed43 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:40.992512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f275b5-02bc-4d4c-97bd-da1343dfed43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.011330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-84f275b5-02bc-4d4c-97bd-da1343dfed43 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:41.012533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-84f275b5-02bc-4d4c-97bd-da1343dfed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:41.064694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:41.075290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 974/1932] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:41.085860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeded9-09be-40dc-a370-7b33c9474ffb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.085860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeded9-09be-40dc-a370-7b33c9474ffb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.085860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeded9-09be-40dc-a370-7b33c9474ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.106744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4efeded9-09be-40dc-a370-7b33c9474ffb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:41.106744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4efeded9-09be-40dc-a370-7b33c9474ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:41.115724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 959/1933] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:41.121925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f275b5-02bc-4d4c-97bd-da1343dfed43 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.141625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f275b5-02bc-4d4c-97bd-da1343dfed43 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.154429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.154429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.154604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.154916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.156958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:41.157464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.159928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.180840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 975/1934] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:48:41.181990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efeded9-09be-40dc-a370-7b33c9474ffb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.194691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efeded9-09be-40dc-a370-7b33c9474ffb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.200086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.200986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.201639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.202021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.203004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.205086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.212830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.215478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:41.235716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb870a-4899-4354-a47f-c293369615af admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.246511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.250764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.251242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.252654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] PATH_INFO: `/v3/users/c65f6f766b594d8b823ecbb7b5fe35e5` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.253862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65f6f766b594d8b823ecbb7b5fe35e5)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.256094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.328720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 960/1935] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:41 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:41.330963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.400453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:41.404239 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d534a9-430d-4edd-9f5a-77d63d1a6805 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.409731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.410561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.410855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.411136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.411864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.413872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.489898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65f6f766b594d8b823ecbb7b5fe35e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.503847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65f6f766b594d8b823ecbb7b5fe35e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.505448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65f6f766b594d8b823ecbb7b5fe35e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.510298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 961/1936] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:41 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:41.511203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.515996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:41.521020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65f6f766b594d8b823ecbb7b5fe35e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.521891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65f6f766b594d8b823ecbb7b5fe35e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.529514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e820a-ce46-45e8-8d09-e2ff9b07e890 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.534314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.535192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.535483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.535767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] PATH_INFO: `/v3/users/60a580a828fa46b99f3a251230ca4d6b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.537012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a580a828fa46b99f3a251230ca4d6b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.538964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.624388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 976/1937] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:41 2022] DELETE /identity/v3/users/c65f6f766b594d8b823ecbb7b5fe35e5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:41.625477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.626871 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:41.696682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c903d-8593-46f0-8d4d-65d223c8e6a7 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.702071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98baba95-1cb8-4695-8a04-51e63074ba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.702767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baba95-1cb8-4695-8a04-51e63074ba52 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.703060 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baba95-1cb8-4695-8a04-51e63074ba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.703341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baba95-1cb8-4695-8a04-51e63074ba52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.704073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98baba95-1cb8-4695-8a04-51e63074ba52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.705724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98baba95-1cb8-4695-8a04-51e63074ba52 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.719812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 977/1938] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:41.719812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031007a3-4063-4869-80e9-e1eb65fc1547 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.719812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031007a3-4063-4869-80e9-e1eb65fc1547 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.719812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031007a3-4063-4869-80e9-e1eb65fc1547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.737049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-031007a3-4063-4869-80e9-e1eb65fc1547 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:41.738271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-031007a3-4063-4869-80e9-e1eb65fc1547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:41.746143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 978/1939] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:41.746939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031007a3-4063-4869-80e9-e1eb65fc1547 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.756977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031007a3-4063-4869-80e9-e1eb65fc1547 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.761748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.762420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.762709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.762995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.764394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.766411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.776781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 979/1940] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:41.777554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.779261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:41.782391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a580a828fa46b99f3a251230ca4d6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.792675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a580a828fa46b99f3a251230ca4d6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.823633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:41.840664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21520f06-9eca-4979-9f1d-a26e3fccd991 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.845725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.846414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.846699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.847002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.856694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.858410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.864261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a580a828fa46b99f3a251230ca4d6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.865022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a580a828fa46b99f3a251230ca4d6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.865818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-adc6a164-e3e6-4ab4-8060-40208ebb1a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a580a828fa46b99f3a251230ca4d6b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:41.868661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 980/1941] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:41.869478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:41.879847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c78473-ea41-4e7c-ae75-347f18437fda service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:41.885622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ace45-21ba-45b5-86f6-2bf7041c8038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:41.886541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ace45-21ba-45b5-86f6-2bf7041c8038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.886809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ace45-21ba-45b5-86f6-2bf7041c8038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.887098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ace45-21ba-45b5-86f6-2bf7041c8038 admin admin] PATH_INFO: `/v3/projects/296bc733552a473280eae2a3ffe9207e/users/ebd699e0fbf94f4a9c45ba0fbc0d39b0/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.911376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 962/1942] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:41 2022] DELETE /identity/v3/users/60a580a828fa46b99f3a251230ca4d6b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:41.914990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ace45-21ba-45b5-86f6-2bf7041c8038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296bc733552a473280eae2a3ffe9207e, user_id=ebd699e0fbf94f4a9c45ba0fbc0d39b0, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:41.917703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:41.918332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:41.918451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ace45-21ba-45b5-86f6-2bf7041c8038 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:41.918863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:41.943979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:41.945690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:42.010892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 963/1943] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:42.030192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.032908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:42.035955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 981/1944] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:41 2022] PUT /identity/v3/projects/296bc733552a473280eae2a3ffe9207e/users/ebd699e0fbf94f4a9c45ba0fbc0d39b0/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:42.048445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.048801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.049072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.086869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] MFA Rules not processed for user `ebd699e0fbf94f4a9c45ba0fbc0d39b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:42.090598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:42.144081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee50be-b037-429e-aa4f-4f2a3cf172ae None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:42.160973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:42.196766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 982/1945] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:42.200783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.204696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:42.210670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:42.211682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.212172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.212764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.284003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e5e2a-8a24-4627-b675-220b8e329c1c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:42.289430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:42.290050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:42.290162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.290387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.290666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.291363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:42.291522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:42.293326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:42.302592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 964/1946] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:42.390998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.392457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:42.419020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 983/1947] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:42 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:42.420037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.424169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2789230-bced-4f35-88d6-1b8cbfebfd23 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:42.512497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:42.513705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.514700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.515117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] PATH_INFO: `/v3/projects/a65b261bba734803b7d29d5bf0bf5abe` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.515710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6f537-3f93-4416-a818-42b312c891f6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:42.517564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65b261bba734803b7d29d5bf0bf5abe)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:42.519819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:42.522505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:42.523206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.523494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.523779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.524481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:42.526582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:42.626031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 984/1948] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:42 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:42.631632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.656780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0e81a-d4eb-43eb-8a02-f59d7ed8e330 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:42.671041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:42.672090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.672090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.672286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.673037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:42.674660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:42.687837 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 985/1949] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:42.696249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.710602 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a188a-9187-4439-b5e4-b38395d87d3e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:42.715164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65b261bba734803b7d29d5bf0bf5abe', 'request_id': 'req-81beecfc-6edc-4092-955b-1ae3dc1c161a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:42.723691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65b261bba734803b7d29d5bf0bf5abe', 'request_id': 'req-81beecfc-6edc-4092-955b-1ae3dc1c161a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:42.783125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:42.784023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.784309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.784592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] PATH_INFO: `/v3/projects/a71a26ee2e124f50bc7e3d5ab6e00161/users/35054279ec2b4507b41172437f7da647/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.800177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71a26ee2e124f50bc7e3d5ab6e00161, user_id=35054279ec2b4507b41172437f7da647, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:42.802961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:42.875873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65b261bba734803b7d29d5bf0bf5abe', 'request_id': 'req-81beecfc-6edc-4092-955b-1ae3dc1c161a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:42.888272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] The token cache is being invalidate because project a65b261bba734803b7d29d5bf0bf5abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:42.888894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:42.890256 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:42.891902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 965/1950] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:42 2022] DELETE /identity/v3/projects/a65b261bba734803b7d29d5bf0bf5abe => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:42.896252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.897631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:42.914306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beecfc-6edc-4092-955b-1ae3dc1c161a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:42.920206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:42.921721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:42.922011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:42.922294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] PATH_INFO: `/v3/users/04db0de1ac4d48efac39be083692a187` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:42.923600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04db0de1ac4d48efac39be083692a187)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:42.925677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:42.966053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 986/1951] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:42 2022] PUT /identity/v3/projects/a71a26ee2e124f50bc7e3d5ab6e00161/users/35054279ec2b4507b41172437f7da647/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:42.966053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:42.968975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:43.051580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3deeb-eca6-4936-8661-108e9ab79c9f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:43.060066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:43.061469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.061755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.062033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] PATH_INFO: `/v3/projects/dca12b0e26544d01b85f5040be80f788` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.063393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca12b0e26544d01b85f5040be80f788)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:43.065485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:43.201908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04db0de1ac4d48efac39be083692a187'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.209705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04db0de1ac4d48efac39be083692a187'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.213551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:43.241165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca12b0e26544d01b85f5040be80f788', 'request_id': 'req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.249590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca12b0e26544d01b85f5040be80f788', 'request_id': 'req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.256130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca12b0e26544d01b85f5040be80f788', 'request_id': 'req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.361728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04db0de1ac4d48efac39be083692a187'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.362531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04db0de1ac4d48efac39be083692a187'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.363350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04db0de1ac4d48efac39be083692a187'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.383821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 966/1952] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:42 2022] DELETE /identity/v3/users/04db0de1ac4d48efac39be083692a187 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:43.385021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:43.386420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:43.460811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c656-5146-4ef6-a826-41a44dc4ff62 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:43.465457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:43.466145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.466438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.466719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.468044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:43.469731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:43.481861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 967/1953] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:43.488264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] The token cache is being invalidate because project dca12b0e26544d01b85f5040be80f788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:43.488564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.489533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:43.489661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-3b60a7cd-bc97-4a19-8dc8-2e228bd2b690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:43.491594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:43.494864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 987/1954] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:42 2022] DELETE /identity/v3/projects/dca12b0e26544d01b85f5040be80f788 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:43.496326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dc05-d2b8-48e1-9cf3-d2f822d72db4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.496635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dc05-d2b8-48e1-9cf3-d2f822d72db4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.496926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dc05-d2b8-48e1-9cf3-d2f822d72db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.506161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e62b53-da10-4651-a863-68062625ef02 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:43.512534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef549b7-b447-40b4-870a-f9cff9653a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:43.515409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef549b7-b447-40b4-870a-f9cff9653a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.515409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef549b7-b447-40b4-870a-f9cff9653a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.515409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef549b7-b447-40b4-870a-f9cff9653a76 admin admin] PATH_INFO: `/v3/projects/a71a26ee2e124f50bc7e3d5ab6e00161/users/35054279ec2b4507b41172437f7da647/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.518002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef549b7-b447-40b4-870a-f9cff9653a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71a26ee2e124f50bc7e3d5ab6e00161, user_id=35054279ec2b4507b41172437f7da647, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:43.520913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef549b7-b447-40b4-870a-f9cff9653a76 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:43.524720 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0c69dc05-d2b8-48e1-9cf3-d2f822d72db4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:43.526367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0c69dc05-d2b8-48e1-9cf3-d2f822d72db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:43.535060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 968/1955] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:43 2022] PUT /identity/v3/projects/a71a26ee2e124f50bc7e3d5ab6e00161/users/35054279ec2b4507b41172437f7da647/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:43.546121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.546425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.551227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.580342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] MFA Rules not processed for user `35054279ec2b4507b41172437f7da647`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:43.581786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:43.598728 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 988/1956] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:43.607049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f3b1-ea27-42e6-87d2-7ef059cc3e5f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.607374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f3b1-ea27-42e6-87d2-7ef059cc3e5f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.607685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f3b1-ea27-42e6-87d2-7ef059cc3e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.630473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8463f3b1-ea27-42e6-87d2-7ef059cc3e5f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:43.632706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8463f3b1-ea27-42e6-87d2-7ef059cc3e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:43.642367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 989/1957] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:43.658407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463f3b1-ea27-42e6-87d2-7ef059cc3e5f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:43.669116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463f3b1-ea27-42e6-87d2-7ef059cc3e5f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:43.675416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 969/1958] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:43.676972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:43.677719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.678076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.678428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.679238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:43.679628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:43.681724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:43.688721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:43.693553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:43.700784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 990/1959] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:48:43.705459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:43.722283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e1832-695f-4cda-a37d-b23fb8d36cba admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:43.726880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:43.727635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.727998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.728351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.728826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:43.729331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:43.731369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:43.781378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed436968-9000-436d-ae5f-b3ad2ec9f545 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:43.786671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:43.788108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.788555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.789421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.806174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:43.808656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:43.822261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 970/1960] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:43.881342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 991/1961] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:43 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:43.886514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:43.902467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf845-360a-481e-9b1c-90b44414f00c service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:43.995113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:43.996124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:43.996627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:43.997171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:43.998121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:44.000807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:44.124590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 971/1962] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:43 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:44.128725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:44.140517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b5429-0897-4898-97d5-db5ca32ddbb8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:44.145783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:44.146792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:44.147298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:44.147792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:44.148774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:44.150597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:44.162548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 992/1963] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:44.167316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:44.171388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:44.175942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:44.193614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc27dd1-21bb-41d4-9e20-8e493889de22 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:44.202788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:44.210466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec487022-f4c8-4891-a53b-ea732336f297 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:44.216440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:44.218166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:44.219001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] PATH_INFO: `/v3/projects/4888b27bb0294677a0bc6c08ff5dc7c5` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:44.220767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4888b27bb0294677a0bc6c08ff5dc7c5)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:44.224705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:44.234280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:44.236921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:44.237156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:44.238914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:44.240386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:44.244583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:44.290823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 972/1964] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:44.292011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:44.310805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb66eb-ca9f-444d-a4ab-f8b114d3efbc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:44.315719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01465a25-8c55-49f8-aa41-a3a5f7b910f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:44.316896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465a25-8c55-49f8-aa41-a3a5f7b910f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:44.321755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465a25-8c55-49f8-aa41-a3a5f7b910f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:44.321755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465a25-8c55-49f8-aa41-a3a5f7b910f3 admin admin] PATH_INFO: `/v3/projects/d551da8f382e4c41aa4892edd084436a/users/e9157a1696e043369b0de1e494a32976/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:44.336281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01465a25-8c55-49f8-aa41-a3a5f7b910f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d551da8f382e4c41aa4892edd084436a, user_id=e9157a1696e043369b0de1e494a32976, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:44.339227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01465a25-8c55-49f8-aa41-a3a5f7b910f3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:44.460390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:44.477805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4888b27bb0294677a0bc6c08ff5dc7c5', 'request_id': 'req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:44.480895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 973/1965] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:44 2022] PUT /identity/v3/projects/d551da8f382e4c41aa4892edd084436a/users/e9157a1696e043369b0de1e494a32976/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:44.484190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363bab0-55c6-4f6c-a257-b9cfc109f268 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:44.484463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363bab0-55c6-4f6c-a257-b9cfc109f268 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:44.484785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363bab0-55c6-4f6c-a257-b9cfc109f268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:44.485756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4888b27bb0294677a0bc6c08ff5dc7c5', 'request_id': 'req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:44.492461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4888b27bb0294677a0bc6c08ff5dc7c5', 'request_id': 'req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:44.511574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3363bab0-55c6-4f6c-a257-b9cfc109f268 None None] MFA Rules not processed for user `e9157a1696e043369b0de1e494a32976`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:44.512823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3363bab0-55c6-4f6c-a257-b9cfc109f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:44.586145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 974/1966] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:44.601132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363bab0-55c6-4f6c-a257-b9cfc109f268 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:44.630565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] The token cache is being invalidate because project 4888b27bb0294677a0bc6c08ff5dc7c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:44.630858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:44.631934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:44.634838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 993/1967] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:44 2022] DELETE /identity/v3/projects/4888b27bb0294677a0bc6c08ff5dc7c5 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:44.646319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:44.648433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:44.725764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363bab0-55c6-4f6c-a257-b9cfc109f268 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:44.734322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:44.735004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:44.735732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:44.736306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:44.752450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:44.768113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:44.772105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:44.786071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfc144-24b8-4f19-b7c6-afdbe04b23b0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:44.789277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 975/1968] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:48:44.805658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:44.806430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:44.806801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:44.807211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] PATH_INFO: `/v3/users/095ca8c4075d40609fa4a8dc6f3efae3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:44.808742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=095ca8c4075d40609fa4a8dc6f3efae3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:44.813841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:45.028343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:45.030693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:45.036262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095ca8c4075d40609fa4a8dc6f3efae3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.051676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af801a56-fd0b-4700-bcb7-9eac251e5de3 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:45.055005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095ca8c4075d40609fa4a8dc6f3efae3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.056131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095ca8c4075d40609fa4a8dc6f3efae3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.061880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:45.063493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:45.063937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:45.065045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] PATH_INFO: `/v3/users/ebd699e0fbf94f4a9c45ba0fbc0d39b0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:45.065168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095ca8c4075d40609fa4a8dc6f3efae3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.065779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5ed3c1c6-f263-4470-869e-36119fb3f853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095ca8c4075d40609fa4a8dc6f3efae3'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.066785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd699e0fbf94f4a9c45ba0fbc0d39b0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:45.070067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:45.185136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 994/1969] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:44 2022] DELETE /identity/v3/users/095ca8c4075d40609fa4a8dc6f3efae3 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:45.376099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd699e0fbf94f4a9c45ba0fbc0d39b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.383851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd699e0fbf94f4a9c45ba0fbc0d39b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.401453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:45.456309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd699e0fbf94f4a9c45ba0fbc0d39b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.457069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd699e0fbf94f4a9c45ba0fbc0d39b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.457699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd699e0fbf94f4a9c45ba0fbc0d39b0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:45.479547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 976/1970] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:45 2022] DELETE /identity/v3/users/ebd699e0fbf94f4a9c45ba0fbc0d39b0 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:45.485078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:45.485777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:45.486329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:45.510293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:45.511654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:45.588533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 995/1971] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:45.595050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:45.596549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:45.673900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a8d3-b65b-4f3c-a77f-a541d8151be1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:45.677760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:45.678572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:45.678863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:45.679138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:45.693736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:45.695281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:45.707019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 977/1972] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:45.869047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:45.874510 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:45.892225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc4e41-bf2b-4735-84af-4a58dbe3b5ef None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:45.898380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:45.899520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:45.899887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:45.902740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] PATH_INFO: `/v3/projects/547f56e7145f44a3bce54a670191dbc2` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:45.902890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547f56e7145f44a3bce54a670191dbc2)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:45.908830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:46.092650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547f56e7145f44a3bce54a670191dbc2', 'request_id': 'req-5f126169-71f8-4cfd-bfbb-c0f106b368a5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.102054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547f56e7145f44a3bce54a670191dbc2', 'request_id': 'req-5f126169-71f8-4cfd-bfbb-c0f106b368a5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.111870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547f56e7145f44a3bce54a670191dbc2', 'request_id': 'req-5f126169-71f8-4cfd-bfbb-c0f106b368a5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.129069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:46.232220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:46.234289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] The token cache is being invalidate because project 547f56e7145f44a3bce54a670191dbc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:46.234855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.235699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5f126169-71f8-4cfd-bfbb-c0f106b368a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.236144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:46.241070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 996/1973] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:45 2022] DELETE /identity/v3/projects/547f56e7145f44a3bce54a670191dbc2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:46.249897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d1f4-8463-412e-9332-a6230e0d6b9d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.249897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d1f4-8463-412e-9332-a6230e0d6b9d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.250307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d1f4-8463-412e-9332-a6230e0d6b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.261439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a645-5a59-4203-99ee-195b2ba487e0 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:46.284479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-e5f9d1f4-8463-412e-9332-a6230e0d6b9d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:46.285648 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e5f9d1f4-8463-412e-9332-a6230e0d6b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:46.334956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:46.335826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.336112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.336391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] PATH_INFO: `/v3/projects/296bc733552a473280eae2a3ffe9207e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.337631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296bc733552a473280eae2a3ffe9207e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:46.339686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:46.369689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 997/1974] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:46.369689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009999-4977-49a8-a11d-e6b823b4a4ca None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.369689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009999-4977-49a8-a11d-e6b823b4a4ca None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.369689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c009999-4977-49a8-a11d-e6b823b4a4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.392201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7c009999-4977-49a8-a11d-e6b823b4a4ca None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:46.393451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7c009999-4977-49a8-a11d-e6b823b4a4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:46.401534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 998/1975] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:46.411205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c009999-4977-49a8-a11d-e6b823b4a4ca None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:46.421481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c009999-4977-49a8-a11d-e6b823b4a4ca None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:46.426159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:46.426803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.427087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.427360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.428075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:46.428376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:46.429995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:46.442218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 999/1976] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:48:46.457054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:46.466587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158c87f-89e9-4e98-9639-ba5b6e0faba7 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:46.472146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:46.472828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.473108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.473417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.474099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:46.476027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:46.523621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296bc733552a473280eae2a3ffe9207e', 'request_id': 'req-4b99b096-3264-4fe0-b009-ed500378f7a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.534618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296bc733552a473280eae2a3ffe9207e', 'request_id': 'req-4b99b096-3264-4fe0-b009-ed500378f7a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.621596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1000/1977] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:46 2022] POST /identity/v3/projects => generated 367 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:46.622442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:46.624641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:46.660222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296bc733552a473280eae2a3ffe9207e', 'request_id': 'req-4b99b096-3264-4fe0-b009-ed500378f7a8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.673138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] The token cache is being invalidate because project 296bc733552a473280eae2a3ffe9207e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:46.673431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.674088 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.675773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 978/1978] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:46 2022] DELETE /identity/v3/projects/296bc733552a473280eae2a3ffe9207e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:46.676700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:46.678008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:46.697835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd16a2-378d-4dec-9905-1be5f1fa731a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:46.702962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:46.703879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.704226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.704498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] PATH_INFO: `/v3/users/1c230254925a43ac9fe52dffbd0fe3ed` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.705848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c230254925a43ac9fe52dffbd0fe3ed)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:46.707836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:46.747092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99b096-3264-4fe0-b009-ed500378f7a8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:46.752875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:46.753585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.753877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.754171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.755147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:46.757295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:46.913090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 979/1979] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:46 2022] POST /identity/v3/users => generated 284 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:46.913090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:46.916651 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:46.933168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919286f7-27ce-424b-a72f-4862256fb8b1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:46.939201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:46.940082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:46.940370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:46.940681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] PATH_INFO: `/v3/users/35054279ec2b4507b41172437f7da647` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:46.941891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35054279ec2b4507b41172437f7da647)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:46.943846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:46.950580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c230254925a43ac9fe52dffbd0fe3ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.963694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c230254925a43ac9fe52dffbd0fe3ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.964425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c230254925a43ac9fe52dffbd0fe3ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.972547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c230254925a43ac9fe52dffbd0fe3ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:46.973261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c230254925a43ac9fe52dffbd0fe3ed'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.071856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1001/1980] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:46 2022] DELETE /identity/v3/users/1c230254925a43ac9fe52dffbd0fe3ed => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:47.072817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:47.074270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:47.143899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02ca4e-e988-4804-95d4-c32890246000 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:47.146853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:47.147591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:47.147934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:47.148218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:47.149107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:47.150780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:47.162594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1002/1981] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:47.168047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:47.179506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b39325-fcab-4ffb-98a2-97d5c0110ba4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:47.186631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:47.187449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:47.187836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:47.188212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:47.189086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:47.190800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:47.202913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1003/1982] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:47.209044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:47.210120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35054279ec2b4507b41172437f7da647'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.218780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35054279ec2b4507b41172437f7da647'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.221201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f3f39-24a4-4a79-9073-e8c3073dfc72 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:47.231078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:47.231519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:47.231797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:47.232074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] PATH_INFO: `/v3/projects/18b35ba7e32c4774a4b0eb7889e941c4/users/bef49100132c4ee1928d5463de2be0a9/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:47.255183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b35ba7e32c4774a4b0eb7889e941c4, user_id=bef49100132c4ee1928d5463de2be0a9, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:47.258112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:47.298188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35054279ec2b4507b41172437f7da647'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.299199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35054279ec2b4507b41172437f7da647'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.299947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9d5b0f30-f87f-40dd-88d1-26cfad86b32d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35054279ec2b4507b41172437f7da647'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.320763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 980/1983] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:46 2022] DELETE /identity/v3/users/35054279ec2b4507b41172437f7da647 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:47.347852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1004/1984] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:47 2022] PUT /identity/v3/projects/18b35ba7e32c4774a4b0eb7889e941c4/users/bef49100132c4ee1928d5463de2be0a9/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:47.360387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:47.360712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:47.360997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:47.382058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:47.384020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:47.414538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] MFA Rules not processed for user `bef49100132c4ee1928d5463de2be0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:47.414538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:47.481425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 981/1985] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:47.494752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:47.495767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:47.504591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06cfe2-5250-4d34-85ea-5704df7bae4d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:47.510533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:47.511650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:47.511921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:47.512278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] PATH_INFO: `/v3/projects/70f83741fb5241efad1b22e93cc67450` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:47.513638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f83741fb5241efad1b22e93cc67450)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:47.516303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:47.568035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:47.599425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35085d6-348c-4a8c-afd7-672c1837f1e0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:47.603487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:47.604143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:47.604465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:47.604786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:47.618298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:47.619792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:47.629436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 982/1986] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:48:47.735094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f83741fb5241efad1b22e93cc67450', 'request_id': 'req-ecb50c34-35a5-4e12-98f5-962e4348503c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.741796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f83741fb5241efad1b22e93cc67450', 'request_id': 'req-ecb50c34-35a5-4e12-98f5-962e4348503c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.753082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f83741fb5241efad1b22e93cc67450', 'request_id': 'req-ecb50c34-35a5-4e12-98f5-962e4348503c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.889479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] The token cache is being invalidate because project 70f83741fb5241efad1b22e93cc67450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:47.892742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.892742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ecb50c34-35a5-4e12-98f5-962e4348503c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:47.893202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1005/1987] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:47 2022] DELETE /identity/v3/projects/70f83741fb5241efad1b22e93cc67450 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:47.894896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:47.896451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:47.899783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79d90d-d94e-44a3-b5fe-100a34c4d4c6 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:47.900402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79d90d-d94e-44a3-b5fe-100a34c4d4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:47.901455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79d90d-d94e-44a3-b5fe-100a34c4d4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:47.923149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021551c-5606-4d58-8e56-18d34020cb18 service glance] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:47.929258 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3b79d90d-d94e-44a3-b5fe-100a34c4d4c6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:47.930555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3b79d90d-d94e-44a3-b5fe-100a34c4d4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:48.000711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.001710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.002005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.002286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] PATH_INFO: `/v3/projects/a71a26ee2e124f50bc7e3d5ab6e00161` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.003593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71a26ee2e124f50bc7e3d5ab6e00161)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.003960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1006/1988] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:48.006585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.013759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7393-f6ce-4689-a395-0993e50d6d07 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.015093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7393-f6ce-4689-a395-0993e50d6d07 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.015625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe7393-f6ce-4689-a395-0993e50d6d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.035700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8efe7393-f6ce-4689-a395-0993e50d6d07 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:48.036945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8efe7393-f6ce-4689-a395-0993e50d6d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:48.045503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1007/1989] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:48.049630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7393-f6ce-4689-a395-0993e50d6d07 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.058461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe7393-f6ce-4689-a395-0993e50d6d07 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.063744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.064387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.064718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.064981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.065694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:48.065997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.067567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.078906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1008/1990] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:48:48.083109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.093833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f8070-0f77-49b7-878f-5ecabe4e5784 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.103745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.103745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.103745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.103745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.104111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.104111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.146853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71a26ee2e124f50bc7e3d5ab6e00161', 'request_id': 'req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:48.153070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71a26ee2e124f50bc7e3d5ab6e00161', 'request_id': 'req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:48.242779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71a26ee2e124f50bc7e3d5ab6e00161', 'request_id': 'req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:48.260700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1009/1991] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/projects => generated 363 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:48.261671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] The token cache is being invalidate because project a71a26ee2e124f50bc7e3d5ab6e00161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:48.261671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:48.261671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1b34ed38-ced7-4e16-8c65-d3ac3ad934dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:48.261892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.261986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 983/1992] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:47 2022] DELETE /identity/v3/projects/a71a26ee2e124f50bc7e3d5ab6e00161 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:48.262057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:48.268295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c0826-4c23-4a01-9ea5-b5e89317c87b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.268727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c0826-4c23-4a01-9ea5-b5e89317c87b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.269112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c0826-4c23-4a01-9ea5-b5e89317c87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.291294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-317c0826-4c23-4a01-9ea5-b5e89317c87b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:48.292529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-317c0826-4c23-4a01-9ea5-b5e89317c87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:48.336500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d463e-62c2-4b6d-aa87-c84a05a34d57 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.343252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.343932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.344216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.344490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.345263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.347196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.353148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 984/1993] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:48.362405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505b7e5-a01f-4b70-b80c-404fd7dda400 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.362850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505b7e5-a01f-4b70-b80c-404fd7dda400 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.363622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505b7e5-a01f-4b70-b80c-404fd7dda400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.382134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f505b7e5-a01f-4b70-b80c-404fd7dda400 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:48.383351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f505b7e5-a01f-4b70-b80c-404fd7dda400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:48.391858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 985/1994] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:48.395959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505b7e5-a01f-4b70-b80c-404fd7dda400 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.405376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505b7e5-a01f-4b70-b80c-404fd7dda400 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.410540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.411208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.411521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.411800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.412543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:48.412917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.414615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.426632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 986/1995] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:48:48.430810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.440410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587d3a1-8fe7-426a-9514-fb0953b63981 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.445240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.445990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.446280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.446624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.447322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.449288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.470022 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1010/1996] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/users => generated 282 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:48.471063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.481074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068edd5-2f20-40e4-9237-20e5fc6d39cf admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.485700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.486347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.486639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.486898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.487692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.489365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.499428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1011/1997] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:48.503190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.514973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f4966-d0de-492b-ac93-b9706354df60 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.519752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.520392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.520709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.520988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.521759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.523355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.534067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1012/1998] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:48.538130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.544823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 987/1999] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:48.548452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6beb-9bfc-4195-8c23-34571255798d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.553366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.622059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74214f-f230-4dae-8032-88c16fb42c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.622849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74214f-f230-4dae-8032-88c16fb42c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.623772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74214f-f230-4dae-8032-88c16fb42c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.623970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74214f-f230-4dae-8032-88c16fb42c02 admin admin] PATH_INFO: `/v3/projects/4b9e93261a3c4e1a8d0f79170713fb9a/users/05c91400909a4c8985c0f48ce0f36263/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.628514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91917c-10be-47e0-af67-694ccf54f043 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.636352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.636352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.636352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.636352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.636753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.638233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.641355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74214f-f230-4dae-8032-88c16fb42c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9e93261a3c4e1a8d0f79170713fb9a, user_id=05c91400909a4c8985c0f48ce0f36263, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.645196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74214f-f230-4dae-8032-88c16fb42c02 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.646923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:48.781259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 988/2000] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:48.781696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1013/2001] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:48 2022] PUT /identity/v3/projects/4b9e93261a3c4e1a8d0f79170713fb9a/users/05c91400909a4c8985c0f48ce0f36263/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:48.782539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.794084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbbb26-eacd-465d-ac50-bec629e74e6f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.794548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbbb26-eacd-465d-ac50-bec629e74e6f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.794699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6ce56-695a-4077-9db1-47e4d875706b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.796026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbbb26-eacd-465d-ac50-bec629e74e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.831521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-57cbbb26-eacd-465d-ac50-bec629e74e6f None None] MFA Rules not processed for user `05c91400909a4c8985c0f48ce0f36263`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:48.832776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-57cbbb26-eacd-465d-ac50-bec629e74e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:48.868022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.868734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.869128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.869328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.870058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.871685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.885114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 989/2002] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:48.886198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.895669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca71d6-969f-450b-8a93-e328b8cee1aa admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.897491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1014/2003] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:48.900741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.901407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.901695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.901801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbbb26-eacd-465d-ac50-bec629e74e6f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.901976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.902719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.904956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.912777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbbb26-eacd-465d-ac50-bec629e74e6f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.917762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 990/2004] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:48.923860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.923860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.923860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.923860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.924221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.924221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:48.925659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:48.936847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60b24c-30d8-4c12-82d5-0a0eebf5c85e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:48.941894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82376125-f67c-4cde-a228-3df6869508aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:48.942943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376125-f67c-4cde-a228-3df6869508aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:48.943229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376125-f67c-4cde-a228-3df6869508aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:48.943512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376125-f67c-4cde-a228-3df6869508aa admin admin] PATH_INFO: `/v3/projects/a9492433bac74b93a08e48857886ff0a/users/84fe41a3bf9a4797a103ba855138a142/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:48.960805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376125-f67c-4cde-a228-3df6869508aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9492433bac74b93a08e48857886ff0a, user_id=84fe41a3bf9a4797a103ba855138a142, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:48.963487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376125-f67c-4cde-a228-3df6869508aa admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.040930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1015/2005] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:48 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:49.043249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.065936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 991/2006] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:48 2022] PUT /identity/v3/projects/a9492433bac74b93a08e48857886ff0a/users/84fe41a3bf9a4797a103ba855138a142/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:49.083249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904e6a5-82a2-4a1a-94ca-afa9561b98d5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.083689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904e6a5-82a2-4a1a-94ca-afa9561b98d5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.084032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904e6a5-82a2-4a1a-94ca-afa9561b98d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.111640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4904e6a5-82a2-4a1a-94ca-afa9561b98d5 None None] MFA Rules not processed for user `84fe41a3bf9a4797a103ba855138a142`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:49.112718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4904e6a5-82a2-4a1a-94ca-afa9561b98d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:49.113874 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee780f-cc06-416d-8367-bb63afcea5ba admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.179248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.182337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.182337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.182337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.182337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.183281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.195174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 992/2007] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:49.199832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904e6a5-82a2-4a1a-94ca-afa9561b98d5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.211329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904e6a5-82a2-4a1a-94ca-afa9561b98d5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.216102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.216818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.217106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.217383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.218081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.220020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.308580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:49.314503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1016/2008] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:49 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:49.315711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 993/2009] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:49 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:49.319937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.320395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.332031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c6e56-3534-4767-8e03-fc3cc789feb2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.393496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810d689-5c33-4ff5-99d7-5c07809215d4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.398972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.399680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.399985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.400272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.401116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.401411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.402172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.402652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.403063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.403316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.403944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.406579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.420748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1017/2010] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:49.426452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.437741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edaa3b-5de8-493d-9d33-d4a0e76eda28 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.442726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.443689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.443981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.444295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] PATH_INFO: `/v3/projects/35480a79084941cdaf70310b7f50a1de/users/cc48e2269920408c8c1393289b4017f0/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.475352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] RBAC: Authorizing `identity:create_grant(project_id=35480a79084941cdaf70310b7f50a1de, user_id=cc48e2269920408c8c1393289b4017f0, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.478420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.493907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 994/2011] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:49 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:49.495470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.507233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79fb80-93bf-4ead-89b1-6a19121fcf49 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.512208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.513472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.513472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.514725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.515496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.518295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.533469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 995/2012] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:49.537749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.547971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b38b-b1b2-4721-91c5-81a8e92f16ad admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.552672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.553510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.553808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.554090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] PATH_INFO: `/v3/projects/dc25481c7e7b40268ec6e61380222576/users/2ace50bc70c84ac4a367ec3f71cbfc50/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.571108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc25481c7e7b40268ec6e61380222576, user_id=2ace50bc70c84ac4a367ec3f71cbfc50, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.573825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.599704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1018/2013] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:49 2022] PUT /identity/v3/projects/35480a79084941cdaf70310b7f50a1de/users/cc48e2269920408c8c1393289b4017f0/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:49.603894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.667101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6535c2-f4e8-4d46-9f10-21e63506657b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.672239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.672940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.673253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.673526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.674248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.676640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.691612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1019/2014] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:49.692504 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.695173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 996/2015] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:49 2022] PUT /identity/v3/projects/dc25481c7e7b40268ec6e61380222576/users/2ace50bc70c84ac4a367ec3f71cbfc50/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:49.703204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf84204-7dfe-471c-a653-e1e257bf6b56 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.704074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.762697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deff334-facf-4d21-a6c0-9ea92d8a15c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.764269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff334-facf-4d21-a6c0-9ea92d8a15c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.764560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff334-facf-4d21-a6c0-9ea92d8a15c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.764867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff334-facf-4d21-a6c0-9ea92d8a15c7 admin admin] PATH_INFO: `/v3/projects/35480a79084941cdaf70310b7f50a1de/users/cc48e2269920408c8c1393289b4017f0/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.766919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deff334-facf-4d21-a6c0-9ea92d8a15c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35480a79084941cdaf70310b7f50a1de, user_id=cc48e2269920408c8c1393289b4017f0, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.769611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deff334-facf-4d21-a6c0-9ea92d8a15c7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.769898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da007abc-a956-45e8-b4ac-ad3b8959dd56 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.774702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.775366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.775709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.775990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.776810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.778411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.780030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1020/2016] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:49 2022] PUT /identity/v3/projects/35480a79084941cdaf70310b7f50a1de/users/cc48e2269920408c8c1393289b4017f0/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:49.789241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 997/2017] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:49.790026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.790323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.792149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.795062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.804642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc032dfc-5091-4afb-b989-1ef6c15926a2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:49.820591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] MFA Rules not processed for user `cc48e2269920408c8c1393289b4017f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:49.821937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:49.888548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52344f95-0f54-4890-a7d4-ecb6cd698378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:49.890161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52344f95-0f54-4890-a7d4-ecb6cd698378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.890728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52344f95-0f54-4890-a7d4-ecb6cd698378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.891153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52344f95-0f54-4890-a7d4-ecb6cd698378 admin admin] PATH_INFO: `/v3/projects/dc25481c7e7b40268ec6e61380222576/users/2ace50bc70c84ac4a367ec3f71cbfc50/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.894129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52344f95-0f54-4890-a7d4-ecb6cd698378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc25481c7e7b40268ec6e61380222576, user_id=2ace50bc70c84ac4a367ec3f71cbfc50, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:49.900525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52344f95-0f54-4890-a7d4-ecb6cd698378 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:49.908679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1021/2018] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:49.922683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 998/2019] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:49 2022] PUT /identity/v3/projects/dc25481c7e7b40268ec6e61380222576/users/2ace50bc70c84ac4a367ec3f71cbfc50/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:49.932354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:49.932679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:49.934776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:49.935237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:49.936104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:49.971091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] MFA Rules not processed for user `2ace50bc70c84ac4a367ec3f71cbfc50`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:49.972334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:50.011879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07afb2-c37d-46db-912d-f964915df89a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:50.055907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 999/2020] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:49 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:50.067879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:50.069534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:50.074799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:50.075396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:50.075679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:50.076508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:50.082962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ef4d-5b75-43ad-b56b-1b9e5f97b824 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:50.086599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:50.087245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:50.087533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:50.087815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:50.097635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:50.099079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:50.108805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1000/2021] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:50 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:48:50.148085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:50.149618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:50.159862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1022/2022] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:50.371520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:50.383634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc904c-a5d5-4182-88c9-1f5d782af6e2 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:50.387040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96865f7-9f71-47b9-8f15-a905f601aaed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:50.387783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96865f7-9f71-47b9-8f15-a905f601aaed service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:50.388074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96865f7-9f71-47b9-8f15-a905f601aaed service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:50.388356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96865f7-9f71-47b9-8f15-a905f601aaed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:50.447501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96865f7-9f71-47b9-8f15-a905f601aaed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:50.449528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96865f7-9f71-47b9-8f15-a905f601aaed service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:50.459676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1001/2023] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:50 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:50.655494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:51.087291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:51.414729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:51.479157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185bca5-2363-4abb-b8cd-358fda08c769 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:51.484825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:51.485715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:51.486032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:51.486314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] PATH_INFO: `/v3/users/84fe41a3bf9a4797a103ba855138a142` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:51.487550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84fe41a3bf9a4797a103ba855138a142)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:51.489579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:51.720508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84fe41a3bf9a4797a103ba855138a142'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:51.732375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84fe41a3bf9a4797a103ba855138a142'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:51.733089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84fe41a3bf9a4797a103ba855138a142'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:51.739986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84fe41a3bf9a4797a103ba855138a142'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:51.742227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84fe41a3bf9a4797a103ba855138a142'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:51.829936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1023/2024] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:51 2022] DELETE /identity/v3/users/84fe41a3bf9a4797a103ba855138a142 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:51.834532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:51.835027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:51.835282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:51.853014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:51.854243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:51.910839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1002/2025] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:51.921106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:51.922592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:51.981740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb70f18-76ed-4e0a-ba27-76d87ba6ee52 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:51.985347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:51.986011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:51.986299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:51.986580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:51.996218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:51.998370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:52.009471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1024/2026] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:52.076948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:52.298878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:52.505183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:52.506743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:52.521314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814789f-cc3a-49d7-8c4f-fbad8cc931d6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:52.526725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:52.527678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:52.527962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:52.530038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] PATH_INFO: `/v3/projects/a9492433bac74b93a08e48857886ff0a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:52.530422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9492433bac74b93a08e48857886ff0a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:52.532476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:52.753127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9492433bac74b93a08e48857886ff0a', 'request_id': 'req-90af7385-0a34-4032-a111-bc2dd43ce116'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:52.760272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9492433bac74b93a08e48857886ff0a', 'request_id': 'req-90af7385-0a34-4032-a111-bc2dd43ce116'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:52.871769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:52.903783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9492433bac74b93a08e48857886ff0a', 'request_id': 'req-90af7385-0a34-4032-a111-bc2dd43ce116'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:52.916974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] The token cache is being invalidate because project a9492433bac74b93a08e48857886ff0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:52.917273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:52.918073 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:52.919806 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1003/2027] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:52 2022] DELETE /identity/v3/projects/a9492433bac74b93a08e48857886ff0a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:52.923816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:52.934030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f30d2-ca05-4a0d-94bf-8008e8f7ae2e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:52.991880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:52.994089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:52.994956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:52.995234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:52.995503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] PATH_INFO: `/v3/users/2ace50bc70c84ac4a367ec3f71cbfc50` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:52.996712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ace50bc70c84ac4a367ec3f71cbfc50)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:52.999254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:53.176321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ace50bc70c84ac4a367ec3f71cbfc50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:53.187955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ace50bc70c84ac4a367ec3f71cbfc50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:53.188655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ace50bc70c84ac4a367ec3f71cbfc50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:53.195464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ace50bc70c84ac4a367ec3f71cbfc50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:53.196095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0c356590-e11f-424f-b733-1c6a55f045db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ace50bc70c84ac4a367ec3f71cbfc50'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:53.329675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1025/2028] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:52 2022] DELETE /identity/v3/users/2ace50bc70c84ac4a367ec3f71cbfc50 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:53.489221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:53.655355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:53.788786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:53.791165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:53.808287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af7385-0a34-4032-a111-bc2dd43ce116 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:53.870968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:53.871749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:53.872032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:53.872309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] PATH_INFO: `/v3/projects/dc25481c7e7b40268ec6e61380222576` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:53.873634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc25481c7e7b40268ec6e61380222576)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:53.875401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:53.875851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:53.982911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc25481c7e7b40268ec6e61380222576', 'request_id': 'req-a139545a-9950-4143-97cf-bebe5f819e00'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:53.989197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc25481c7e7b40268ec6e61380222576', 'request_id': 'req-a139545a-9950-4143-97cf-bebe5f819e00'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.083449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:54.114907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc25481c7e7b40268ec6e61380222576', 'request_id': 'req-a139545a-9950-4143-97cf-bebe5f819e00'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.131106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] The token cache is being invalidate because project dc25481c7e7b40268ec6e61380222576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:54.131106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.131106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a139545a-9950-4143-97cf-bebe5f819e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.131106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1004/2029] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:53 2022] DELETE /identity/v3/projects/dc25481c7e7b40268ec6e61380222576 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:54.134651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a794-c735-41d2-bd68-43ce7160e2b4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.134950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a794-c735-41d2-bd68-43ce7160e2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.135236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2a794-c735-41d2-bd68-43ce7160e2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.156822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0ca2a794-c735-41d2-bd68-43ce7160e2b4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:54.159668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0ca2a794-c735-41d2-bd68-43ce7160e2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:54.220960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1026/2030] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:54.233342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e412ca-0677-4d9c-b44b-ad33e35dabe1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.233751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e412ca-0677-4d9c-b44b-ad33e35dabe1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.233751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e412ca-0677-4d9c-b44b-ad33e35dabe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.253695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-10e412ca-0677-4d9c-b44b-ad33e35dabe1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:54.255176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-10e412ca-0677-4d9c-b44b-ad33e35dabe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:54.263283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1005/2031] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:54.267955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a794-c735-41d2-bd68-43ce7160e2b4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:54.277697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2a794-c735-41d2-bd68-43ce7160e2b4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:54.283151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:54.283811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.284100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.284380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.285144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:54.285455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:54.287069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:54.299223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1027/2032] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:48:54.303700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e412ca-0677-4d9c-b44b-ad33e35dabe1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:54.313985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e412ca-0677-4d9c-b44b-ad33e35dabe1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:54.322825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:54.323564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.323857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.324137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.324889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:54.337213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:54.418313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:54.420189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:54.434940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1006/2033] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:54 2022] POST /identity/v3/projects => generated 399 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:54.439784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:54.445321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea702e-b0a9-4c8f-bab5-5e0e5b37d585 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:54.508569 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:54.509517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.509836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.510065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] PATH_INFO: `/v3/users/05c91400909a4c8985c0f48ce0f36263` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.511404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c91400909a4c8985c0f48ce0f36263)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:54.513354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:54.516183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfa130-26b0-4d9f-8224-e9dbaeb4b78e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:54.521096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:54.521851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.522140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.522415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.523108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:54.525070 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:54.738604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1007/2034] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:54 2022] POST /identity/v3/users => generated 300 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:54.739877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:54.750259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092d639-7a17-4d32-9e81-13c16fcd2884 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:54.755846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:54.756709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.757000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.757280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.758043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:54.759679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:54.770708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1008/2035] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:54.774613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:54.785060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc2fee-0b66-4753-9384-0101cf179823 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:54.789799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:54.790566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.790854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.791131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.791907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:54.793549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:54.804232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1009/2036] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:54.808213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:54.817968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f51fe1-3518-40e2-9170-121df648d45a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:54.822721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df82bdf-0640-4ad2-8125-353f338e3ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:54.823582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df82bdf-0640-4ad2-8125-353f338e3ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.823869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df82bdf-0640-4ad2-8125-353f338e3ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.824198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df82bdf-0640-4ad2-8125-353f338e3ce7 admin admin] PATH_INFO: `/v3/projects/2ffebc3ea95c4ecfbd33bb6fe095bc07/users/1582688a5d3f49bb88498a2b8140a796/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.839366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df82bdf-0640-4ad2-8125-353f338e3ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffebc3ea95c4ecfbd33bb6fe095bc07, user_id=1582688a5d3f49bb88498a2b8140a796, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:54.842739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df82bdf-0640-4ad2-8125-353f338e3ce7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:54.859430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c91400909a4c8985c0f48ce0f36263'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.872591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c91400909a4c8985c0f48ce0f36263'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.873941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c91400909a4c8985c0f48ce0f36263'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.881505 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c91400909a4c8985c0f48ce0f36263'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.882118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9dbfc0b9-112c-4991-a540-636c154dadd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c91400909a4c8985c0f48ce0f36263'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:54.894342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:54.929428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1010/2037] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:54 2022] PUT /identity/v3/projects/2ffebc3ea95c4ecfbd33bb6fe095bc07/users/1582688a5d3f49bb88498a2b8140a796/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:54.940071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.940372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.940685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.963604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1028/2038] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:54 2022] DELETE /identity/v3/users/05c91400909a4c8985c0f48ce0f36263 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:54.968045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:54.968335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:54.968621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:54.977033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] MFA Rules not processed for user `1582688a5d3f49bb88498a2b8140a796`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:54.978254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:54.986646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:54.987969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:55.055999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1029/2039] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:55.065604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1011/2040] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:48:55.069170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:55.069639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:55.070822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:55.071032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:55.085497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742028a-b40c-46db-af79-0d805c9c6c55 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:55.090448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:55.091121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:55.091405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:55.092317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:55.092737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:55.094652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:55.131693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907a293-83d5-40ee-9afe-192760b9b1b3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:55.135901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:55.136557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:55.136879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:55.137157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:55.150917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:55.151577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:55.164293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1030/2041] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:55.248630 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1012/2042] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:55 2022] POST /identity/v3/projects => generated 399 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:48:55.253747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:55.267268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:55.270881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4c686-4c57-4eb8-bf61-2eef6dc3ae89 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:55.345157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:55.345852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:55.346944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:55.346944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:55.347126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:55.349143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:55.777801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1031/2043] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:55 2022] POST /identity/v3/users => generated 306 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:55.780354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:55.790399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498952a-f7c4-4b36-b4e2-be8e3e8309dd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:55.795670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:55.796364 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:55.796737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:55.797434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:55.798017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:55.799822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:55.815106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1013/2044] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:48:55.815883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:55.828138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5a8b0-585f-4f2e-a4a2-ec7069683059 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:55.833159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:55.834071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:55.834358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:55.834641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] PATH_INFO: `/v3/projects/a7b9accacd71410c8257250621d4f7ee/users/c43e2fed05454274a09d821e03164fd1/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:55.851187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b9accacd71410c8257250621d4f7ee, user_id=c43e2fed05454274a09d821e03164fd1, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:55.853961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:55.959083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1032/2045] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:55 2022] PUT /identity/v3/projects/a7b9accacd71410c8257250621d4f7ee/users/c43e2fed05454274a09d821e03164fd1/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:55.963016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:55.974735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e495bf-1b7a-46fc-9e8c-2cac9d9e026f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:56.041119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df40fff-037f-445b-a7f8-9a818ce4c538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:56.041752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df40fff-037f-445b-a7f8-9a818ce4c538 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:56.042115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df40fff-037f-445b-a7f8-9a818ce4c538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:56.042279 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df40fff-037f-445b-a7f8-9a818ce4c538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:56.043032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df40fff-037f-445b-a7f8-9a818ce4c538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:56.044848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df40fff-037f-445b-a7f8-9a818ce4c538 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:56.056145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1014/2046] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:56.060368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:56.071451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fb8a8-f20e-485c-a75e-4528080ae5fa admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:56.077404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:56.078389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:56.078736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:56.079094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] PATH_INFO: `/v3/projects/a7b9accacd71410c8257250621d4f7ee/users/c43e2fed05454274a09d821e03164fd1/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:56.081195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b9accacd71410c8257250621d4f7ee, user_id=c43e2fed05454274a09d821e03164fd1, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:56.084041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:56.096195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1033/2047] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:56 2022] PUT /identity/v3/projects/a7b9accacd71410c8257250621d4f7ee/users/c43e2fed05454274a09d821e03164fd1/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:56.108099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:56.114980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:56.114980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:56.115224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:56.140944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] MFA Rules not processed for user `c43e2fed05454274a09d821e03164fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:56.142245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:56.225786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1015/2048] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:56.232029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:56.238812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:56.241237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:56.313028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c874b46-6a41-40b2-93d6-73cf544edf3e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:56.317274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:56.317941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:56.318224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:56.318500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:56.329687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:56.331137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:56.341330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1034/2049] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:48:56.383737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:56.383881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:56.453800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d8283-00d1-42c6-bf19-c7b61383ca95 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:56.463665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:56.463665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:56.463665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:56.463665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] PATH_INFO: `/v3/projects/4b9e93261a3c4e1a8d0f79170713fb9a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:56.464308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9e93261a3c4e1a8d0f79170713fb9a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:56.464308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:56.658113 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:56.658643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9e93261a3c4e1a8d0f79170713fb9a', 'request_id': 'req-03584605-cc81-4cf8-95cc-b4fc4aaa518a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:56.659585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:56.665025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9e93261a3c4e1a8d0f79170713fb9a', 'request_id': 'req-03584605-cc81-4cf8-95cc-b4fc4aaa518a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:56.668747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:56.672842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077c141-99cd-408b-9359-dff076401af7 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:56.723485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:56.724146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:56.724435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:56.724762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:56.755476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9e93261a3c4e1a8d0f79170713fb9a', 'request_id': 'req-03584605-cc81-4cf8-95cc-b4fc4aaa518a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:56.766486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] The token cache is being invalidate because project 4b9e93261a3c4e1a8d0f79170713fb9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:56.766777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:56.767466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:56.769254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1016/2050] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:56 2022] DELETE /identity/v3/projects/4b9e93261a3c4e1a8d0f79170713fb9a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:56.772863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:56.774198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:56.783211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:56.784783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:56.795289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1035/2051] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:48:56.842860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584605-cc81-4cf8-95cc-b4fc4aaa518a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:56.847773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:56.849855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:56.849855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:56.849855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] PATH_INFO: `/v3/users/cc48e2269920408c8c1393289b4017f0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:56.850638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc48e2269920408c8c1393289b4017f0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:56.852652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:57.131473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc48e2269920408c8c1393289b4017f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:57.138357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc48e2269920408c8c1393289b4017f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:57.237690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc48e2269920408c8c1393289b4017f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:57.238415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc48e2269920408c8c1393289b4017f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:57.239208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-e40bec30-eab2-446f-a599-d1becc4d5736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc48e2269920408c8c1393289b4017f0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:57.254648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:57.265966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1017/2052] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:48:56 2022] DELETE /identity/v3/users/cc48e2269920408c8c1393289b4017f0 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:48:57.647469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:57.810189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:57.824203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbe47-f4ea-45f5-9ee9-d055069d82c0 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:57.890762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:57.891617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:57.891898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:57.892172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] PATH_INFO: `/v3/projects/35480a79084941cdaf70310b7f50a1de` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:57.893374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35480a79084941cdaf70310b7f50a1de)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:57.895401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:57.995485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35480a79084941cdaf70310b7f50a1de', 'request_id': 'req-586f1791-2b33-48bd-bf3f-e596576bb478'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:58.002639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35480a79084941cdaf70310b7f50a1de', 'request_id': 'req-586f1791-2b33-48bd-bf3f-e596576bb478'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:58.009106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35480a79084941cdaf70310b7f50a1de', 'request_id': 'req-586f1791-2b33-48bd-bf3f-e596576bb478'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:58.071550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:58.116747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] The token cache is being invalidate because project 35480a79084941cdaf70310b7f50a1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:48:58.119711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:58.119711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-586f1791-2b33-48bd-bf3f-e596576bb478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:48:58.119711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1036/2053] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:48:57 2022] DELETE /identity/v3/projects/35480a79084941cdaf70310b7f50a1de => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:58.125058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e745c-d14e-4914-b68a-84cb06797ec9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.125350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e745c-d14e-4914-b68a-84cb06797ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.125684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e745c-d14e-4914-b68a-84cb06797ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.146895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a27e745c-d14e-4914-b68a-84cb06797ec9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:58.150661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a27e745c-d14e-4914-b68a-84cb06797ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:58.206858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1018/2054] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:58.216057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406314f1-eba8-4a2c-86d5-a1dd9fd5c86a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.216406 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406314f1-eba8-4a2c-86d5-a1dd9fd5c86a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.217221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406314f1-eba8-4a2c-86d5-a1dd9fd5c86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.235628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-406314f1-eba8-4a2c-86d5-a1dd9fd5c86a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:58.236912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-406314f1-eba8-4a2c-86d5-a1dd9fd5c86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:58.249817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1037/2055] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:58.254461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e745c-d14e-4914-b68a-84cb06797ec9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:58.264846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e745c-d14e-4914-b68a-84cb06797ec9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:58.270569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:58.271409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.271735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.271973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.272738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:48:58.273045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:58.274737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:58.287136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1019/2056] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:48:58.291594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406314f1-eba8-4a2c-86d5-a1dd9fd5c86a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:58.300980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406314f1-eba8-4a2c-86d5-a1dd9fd5c86a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:58.305735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:58.306383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.306763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.306930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.307627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:58.309598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:58.446741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1038/2057] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:58 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:58.449812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:58.460955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abae25d-375f-495d-ac92-eb1fd6722559 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:58.473604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:58.522985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:58.523676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.523967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.524248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.525119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:58.527094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:58.652761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:58.707864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1020/2058] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:58 2022] POST /identity/v3/users => generated 284 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:58.712530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:58.721672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ff562-0397-4778-bd53-5df364b86573 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:58.726747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:58.727429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.727943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.728071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.728743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:58.730372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:58.742527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1039/2059] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:58.746421 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:58.757391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4738-8c1f-4836-92ae-3b34f3990b8f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:58.762046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108688d5-3abe-430e-b976-fc2879d134c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:58.762750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108688d5-3abe-430e-b976-fc2879d134c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.763156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108688d5-3abe-430e-b976-fc2879d134c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.764028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108688d5-3abe-430e-b976-fc2879d134c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.764816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108688d5-3abe-430e-b976-fc2879d134c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:58.766460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108688d5-3abe-430e-b976-fc2879d134c4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:58.780417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1021/2060] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:58.781788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:58.792560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9ae73-42f3-4999-937c-b5bc021a2b8b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:58.797278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:58.799868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.799868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.799868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] PATH_INFO: `/v3/projects/10c698a6962547d186f4dc7599462605/users/22785e01d7e24f2f84a71c98f4bc7cb8/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.816162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c698a6962547d186f4dc7599462605, user_id=22785e01d7e24f2f84a71c98f4bc7cb8, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:58.819040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:58.924719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1040/2061] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:58 2022] PUT /identity/v3/projects/10c698a6962547d186f4dc7599462605/users/22785e01d7e24f2f84a71c98f4bc7cb8/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:58.935719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9b81-ab95-4fa1-aec1-944f5c2d330e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:58.936186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9b81-ab95-4fa1-aec1-944f5c2d330e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:58.936470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9b81-ab95-4fa1-aec1-944f5c2d330e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:58.965679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ca0d9b81-ab95-4fa1-aec1-944f5c2d330e None None] MFA Rules not processed for user `22785e01d7e24f2f84a71c98f4bc7cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:58.967135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ca0d9b81-ab95-4fa1-aec1-944f5c2d330e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:59.033287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1022/2062] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:59.037850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:59.099784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaa146-fff8-4fcb-a6ab-896b4c64de61 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:59.104819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:59.105751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.105991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.106371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.107709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:59.113453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:59.215421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1041/2063] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:48:59 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:59.220125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d9b81-ab95-4fa1-aec1-944f5c2d330e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:59.231301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d9b81-ab95-4fa1-aec1-944f5c2d330e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:59.291768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:59.292493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.292799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.293073 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.293768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:59.295808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:59.311518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:59.378237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1023/2064] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:48:59 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:48:59.380271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:59.390961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9bb0f-acad-43ae-b85a-eff7c9dfa564 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:59.395850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:59.396530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.396855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.397141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.397873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:59.399465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:59.401559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:48:59.410770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1042/2065] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:59.421272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:59.439849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c0ef-902a-42ef-baf5-86f3c25a3e5f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:59.450132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:59.452659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.452779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.453172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] PATH_INFO: `/v3/projects/e412de340ed84b3688ab4fba22c40eba/users/5905759ec39b4e2797f7181814317e4b/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.475191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e412de340ed84b3688ab4fba22c40eba, user_id=5905759ec39b4e2797f7181814317e4b, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:59.478542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:59.578453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1024/2066] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:59 2022] PUT /identity/v3/projects/e412de340ed84b3688ab4fba22c40eba/users/5905759ec39b4e2797f7181814317e4b/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:59.578627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:59.586042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cb9a6-5b3b-4f04-b900-ba89d07ffe73 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:59.650805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecd04e-0e41-459e-9e16-69f33f53b02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:59.651484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecd04e-0e41-459e-9e16-69f33f53b02d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.651903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecd04e-0e41-459e-9e16-69f33f53b02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.652325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecd04e-0e41-459e-9e16-69f33f53b02d admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.653263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecd04e-0e41-459e-9e16-69f33f53b02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:59.658694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecd04e-0e41-459e-9e16-69f33f53b02d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:59.665797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1043/2067] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:48:59.676274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:59.695108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94df3ad-8d4d-4678-85ef-aaf5b889fd13 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:59.700474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:59.702163 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.702529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.702894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] PATH_INFO: `/v3/projects/e412de340ed84b3688ab4fba22c40eba/users/5905759ec39b4e2797f7181814317e4b/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.705264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e412de340ed84b3688ab4fba22c40eba, user_id=5905759ec39b4e2797f7181814317e4b, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:59.710711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:59.720650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1025/2068] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:48:59 2022] PUT /identity/v3/projects/e412de340ed84b3688ab4fba22c40eba/users/5905759ec39b4e2797f7181814317e4b/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:48:59.732587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329176-df8c-484f-8e82-d5e4801ac2f9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.733092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329176-df8c-484f-8e82-d5e4801ac2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.739065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329176-df8c-484f-8e82-d5e4801ac2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.767801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fb329176-df8c-484f-8e82-d5e4801ac2f9 None None] MFA Rules not processed for user `5905759ec39b4e2797f7181814317e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:48:59.769196 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fb329176-df8c-484f-8e82-d5e4801ac2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:59.844732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1044/2069] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:48:59 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:48:59.855990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:48:59.858315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:48:59.919489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83805-ced6-41b2-8af3-c4a49716c4b8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:48:59.923337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:48:59.923995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:48:59.924285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:48:59.924568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:48:59.933622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:48:59.935114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:48:59.944458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1026/2070] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:48:59 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:00.119898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329176-df8c-484f-8e82-d5e4801ac2f9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:00.130233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329176-df8c-484f-8e82-d5e4801ac2f9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:00.134386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b0fd-65d4-40e0-98ed-148a9613ddec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:00.139736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b0fd-65d4-40e0-98ed-148a9613ddec service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:00.139736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b0fd-65d4-40e0-98ed-148a9613ddec service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:00.139736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b0fd-65d4-40e0-98ed-148a9613ddec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:00.198347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b0fd-65d4-40e0-98ed-148a9613ddec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:00.199793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b0fd-65d4-40e0-98ed-148a9613ddec service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:00.209533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1045/2071] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:00.224717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:00.969960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:01.483757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:02.666347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:02.668107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:02.740394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e942fae-6d10-482f-b01f-3e187879b16e service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:02.745020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:02.747266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:02.747266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:02.747266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] PATH_INFO: `/v3/users/1582688a5d3f49bb88498a2b8140a796` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:02.747656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1582688a5d3f49bb88498a2b8140a796)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:02.751458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:02.966519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1582688a5d3f49bb88498a2b8140a796'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:02.974214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1582688a5d3f49bb88498a2b8140a796'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:03.094415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1582688a5d3f49bb88498a2b8140a796'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:03.095036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1582688a5d3f49bb88498a2b8140a796'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:03.095694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1582688a5d3f49bb88498a2b8140a796'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:03.114092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1027/2072] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:02 2022] DELETE /identity/v3/users/1582688a5d3f49bb88498a2b8140a796 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:03.118694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:03.119226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:03.119323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:03.137634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:03.138850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:03.208214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1046/2073] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:03.217742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:03.219136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:03.294571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2c079-df6e-4195-bba3-3722cecdbd51 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:03.303009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:03.303009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:03.303009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:03.303009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:03.313788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:03.316974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:03.354923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1028/2074] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:03.946735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:03.949161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:03.967463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a8f47-3f54-4f40-aed3-f1a0f4fe811f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:03.972627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:03.973567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:03.973852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:03.974130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] PATH_INFO: `/v3/projects/2ffebc3ea95c4ecfbd33bb6fe095bc07` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:03.975343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffebc3ea95c4ecfbd33bb6fe095bc07)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:03.977313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:04.205053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffebc3ea95c4ecfbd33bb6fe095bc07', 'request_id': 'req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.211471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffebc3ea95c4ecfbd33bb6fe095bc07', 'request_id': 'req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.217416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffebc3ea95c4ecfbd33bb6fe095bc07', 'request_id': 'req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.300335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] The token cache is being invalidate because project 2ffebc3ea95c4ecfbd33bb6fe095bc07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:04.300759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.301859 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.304197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1047/2075] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:03 2022] DELETE /identity/v3/projects/2ffebc3ea95c4ecfbd33bb6fe095bc07 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:04.308059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:04.319780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec500963-c34e-4dcd-b77c-457bc0d47b54 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:04.332581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:04.381092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:04.382011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:04.382308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:04.382732 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] PATH_INFO: `/v3/users/c43e2fed05454274a09d821e03164fd1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:04.384006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c43e2fed05454274a09d821e03164fd1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:04.385994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:04.620512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c43e2fed05454274a09d821e03164fd1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.627964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c43e2fed05454274a09d821e03164fd1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.751379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c43e2fed05454274a09d821e03164fd1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.751995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c43e2fed05454274a09d821e03164fd1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.752687 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5ed6d222-ae80-4a35-9884-d1e3373b981a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c43e2fed05454274a09d821e03164fd1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:04.771081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1029/2076] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:04 2022] DELETE /identity/v3/users/c43e2fed05454274a09d821e03164fd1 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:05.268660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:05.270439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:05.305680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3b02b-255e-4fcb-8e4a-a622c65282a5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:05.381257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:05.388104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:05.389013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:05.389361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:05.389716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] PATH_INFO: `/v3/projects/a7b9accacd71410c8257250621d4f7ee` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:05.392001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7b9accacd71410c8257250621d4f7ee)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:05.394449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:05.534495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7b9accacd71410c8257250621d4f7ee', 'request_id': 'req-ebea3e17-40af-4052-8d56-b21abb0173b4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:05.540939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7b9accacd71410c8257250621d4f7ee', 'request_id': 'req-ebea3e17-40af-4052-8d56-b21abb0173b4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:05.546836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7b9accacd71410c8257250621d4f7ee', 'request_id': 'req-ebea3e17-40af-4052-8d56-b21abb0173b4'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:05.673479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] The token cache is being invalidate because project a7b9accacd71410c8257250621d4f7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:05.674068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:05.674745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-ebea3e17-40af-4052-8d56-b21abb0173b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:05.676862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1048/2077] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:05 2022] DELETE /identity/v3/projects/a7b9accacd71410c8257250621d4f7ee => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:05.681543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad958d-c9a5-4197-ae6f-37b1f95feba9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:05.682030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad958d-c9a5-4197-ae6f-37b1f95feba9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:05.682125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad958d-c9a5-4197-ae6f-37b1f95feba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:05.709580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-31ad958d-c9a5-4197-ae6f-37b1f95feba9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:05.711424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-31ad958d-c9a5-4197-ae6f-37b1f95feba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:05.790800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1030/2078] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:05.799341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225cf8-d595-455b-82d3-7287aeee1b1f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:05.799775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225cf8-d595-455b-82d3-7287aeee1b1f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:05.800299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225cf8-d595-455b-82d3-7287aeee1b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:05.820819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-04225cf8-d595-455b-82d3-7287aeee1b1f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:05.822095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-04225cf8-d595-455b-82d3-7287aeee1b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:05.830923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1049/2079] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:05.838925 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad958d-c9a5-4197-ae6f-37b1f95feba9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:05.850002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad958d-c9a5-4197-ae6f-37b1f95feba9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:05.859099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:05.859099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:05.859099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:05.859099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:05.859751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:05.859751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:05.859751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:05.874521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1031/2080] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:49:05.875716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04225cf8-d595-455b-82d3-7287aeee1b1f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:05.886326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04225cf8-d595-455b-82d3-7287aeee1b1f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:05.891657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:05.892385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:05.892586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:05.892905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:05.893592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:05.895494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.025948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1050/2081] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:05 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:06.027151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.034900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.036585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:06.041487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bc89c-2ee4-4615-bac9-fffa65f279a1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.064431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:06.121538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.123321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.123605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.125730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.125730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.127184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.129826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237563e-0b5b-4212-8159-6a62bd3b43a7 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.136981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.139413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.139808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.140837 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] PATH_INFO: `/v3/users/22785e01d7e24f2f84a71c98f4bc7cb8` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.142567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] RBAC: Authorizing `identity:delete_user(user_id=22785e01d7e24f2f84a71c98f4bc7cb8)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.144940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.246506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1032/2082] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:06 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:06.246905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.258396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0104d47-cd94-459d-ae6b-f26b337f17ca admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.263187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.263849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.264170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.264457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.265800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.267434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.277274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1033/2083] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:49:06.282018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.294743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d5a-997d-44c4-ad0f-9301802ff977 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.300620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.301304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.301592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.301973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.302589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.304218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.315418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1034/2084] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:49:06.319296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.331299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904d77-ec15-4874-914b-f8ab3fea959e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.336337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9c145-cfc8-46b7-b78d-b7050f46b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.341231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9c145-cfc8-46b7-b78d-b7050f46b30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.341231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9c145-cfc8-46b7-b78d-b7050f46b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.341231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9c145-cfc8-46b7-b78d-b7050f46b30d admin admin] PATH_INFO: `/v3/projects/e25aa2225d084115b3ce50f824dac3b7/users/813a85a174984206ab78c7b83c5838b9/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.355192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9c145-cfc8-46b7-b78d-b7050f46b30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e25aa2225d084115b3ce50f824dac3b7, user_id=813a85a174984206ab78c7b83c5838b9, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.358130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9c145-cfc8-46b7-b78d-b7050f46b30d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.376662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22785e01d7e24f2f84a71c98f4bc7cb8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:06.387697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:06.390481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22785e01d7e24f2f84a71c98f4bc7cb8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:06.391603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22785e01d7e24f2f84a71c98f4bc7cb8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:06.400795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22785e01d7e24f2f84a71c98f4bc7cb8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:06.401676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cf181711-6e19-41d4-99b1-08b344c8f17a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22785e01d7e24f2f84a71c98f4bc7cb8'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:06.442700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1035/2085] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:06 2022] PUT /identity/v3/projects/e25aa2225d084115b3ce50f824dac3b7/users/813a85a174984206ab78c7b83c5838b9/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:06.453560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.453842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.454174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.487710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] MFA Rules not processed for user `813a85a174984206ab78c7b83c5838b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:06.489087 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:06.526101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1051/2086] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:06 2022] DELETE /identity/v3/users/22785e01d7e24f2f84a71c98f4bc7cb8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:06.531614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454347eb-667b-4584-92d0-a137a3af56de None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.532093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454347eb-667b-4584-92d0-a137a3af56de None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.532476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454347eb-667b-4584-92d0-a137a3af56de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.557491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-454347eb-667b-4584-92d0-a137a3af56de None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:06.558745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-454347eb-667b-4584-92d0-a137a3af56de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:06.579329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1036/2087] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:06.590378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.591989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:06.663422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1052/2088] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:06.678053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a831d7f-2090-456b-ac91-bfb99fd06c3b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.680493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454347eb-667b-4584-92d0-a137a3af56de None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.682162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-454347eb-667b-4584-92d0-a137a3af56de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:06.686977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:06.732977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313b9c-0289-43d7-8f66-4027bbd407e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.733747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313b9c-0289-43d7-8f66-4027bbd407e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.734649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313b9c-0289-43d7-8f66-4027bbd407e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.735120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313b9c-0289-43d7-8f66-4027bbd407e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.749520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454347eb-667b-4584-92d0-a137a3af56de None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.755369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.756279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.759382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.759382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.769802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.771419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.785305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1053/2089] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:06.797159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313b9c-0289-43d7-8f66-4027bbd407e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.798451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313b9c-0289-43d7-8f66-4027bbd407e3 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:06.807607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1037/2090] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:06.827917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:06.829879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:06.850176 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8326998-f4c3-49a1-9f53-5fd0cb84044f service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:06.859878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:06.861230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:06.861778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:06.869636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] PATH_INFO: `/v3/users/813a85a174984206ab78c7b83c5838b9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:06.881761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813a85a174984206ab78c7b83c5838b9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:06.882466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:07.150054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813a85a174984206ab78c7b83c5838b9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.168125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813a85a174984206ab78c7b83c5838b9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.169321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813a85a174984206ab78c7b83c5838b9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.179060 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813a85a174984206ab78c7b83c5838b9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.179782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813a85a174984206ab78c7b83c5838b9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.182850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:07.291860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1054/2091] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:06 2022] DELETE /identity/v3/users/813a85a174984206ab78c7b83c5838b9 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:07.296422 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:07.296768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:07.297132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:07.313803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:07.314948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:07.355567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:07.357717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:07.410524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1038/2092] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:07.420655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:07.422619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:07.472467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbc406-53cc-450d-91cc-ecfca9066855 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:07.479228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:07.480114 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:07.480401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:07.480712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] PATH_INFO: `/v3/projects/10c698a6962547d186f4dc7599462605` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:07.481946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c698a6962547d186f4dc7599462605)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:07.484421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:07.538665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29057621-5f40-413c-8a88-64c1d4db5092 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:07.550639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:07.552043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:07.552418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:07.552742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:07.573402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:07.575355 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:07.597901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1039/2093] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:07.668416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c698a6962547d186f4dc7599462605', 'request_id': 'req-6238e45f-87a0-40fa-a9af-d0e46432d8de'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.676305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c698a6962547d186f4dc7599462605', 'request_id': 'req-6238e45f-87a0-40fa-a9af-d0e46432d8de'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.683649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c698a6962547d186f4dc7599462605', 'request_id': 'req-6238e45f-87a0-40fa-a9af-d0e46432d8de'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.797255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] The token cache is being invalidate because project 10c698a6962547d186f4dc7599462605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:07.797740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.798402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:07.800079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1055/2094] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:07 2022] DELETE /identity/v3/projects/10c698a6962547d186f4dc7599462605 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:07.804491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:07.814972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797572b4-7ade-4a39-9e95-c582edfb9f8d service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:07.820119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:07.895469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:07.897417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:07.898222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:07.909784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] PATH_INFO: `/v3/users/5905759ec39b4e2797f7181814317e4b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:07.909784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5905759ec39b4e2797f7181814317e4b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:07.909784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:08.148742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:08.150232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:08.177291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238e45f-87a0-40fa-a9af-d0e46432d8de admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:08.177291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:08.177745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:08.177883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:08.178692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] PATH_INFO: `/v3/projects/e25aa2225d084115b3ce50f824dac3b7` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:08.190949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5905759ec39b4e2797f7181814317e4b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.196688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e25aa2225d084115b3ce50f824dac3b7)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:08.198697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:08.201439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5905759ec39b4e2797f7181814317e4b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.344848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5905759ec39b4e2797f7181814317e4b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.345602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5905759ec39b4e2797f7181814317e4b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.346283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-11d31fad-1338-4329-b73f-4f18e822d449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5905759ec39b4e2797f7181814317e4b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.365375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e25aa2225d084115b3ce50f824dac3b7', 'request_id': 'req-aa6c9491-3b67-4922-9acf-1f63140d99cb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.371840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e25aa2225d084115b3ce50f824dac3b7', 'request_id': 'req-aa6c9491-3b67-4922-9acf-1f63140d99cb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.377614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e25aa2225d084115b3ce50f824dac3b7', 'request_id': 'req-aa6c9491-3b67-4922-9acf-1f63140d99cb'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.378911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1040/2095] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:07 2022] DELETE /identity/v3/users/5905759ec39b4e2797f7181814317e4b => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:08.572824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] The token cache is being invalidate because project e25aa2225d084115b3ce50f824dac3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:08.573314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.573838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa6c9491-3b67-4922-9acf-1f63140d99cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:08.575638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1056/2096] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:08 2022] DELETE /identity/v3/projects/e25aa2225d084115b3ce50f824dac3b7 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:08.581362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827eada-72bb-4c1b-bd72-7eb79fce21cd None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:08.583370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827eada-72bb-4c1b-bd72-7eb79fce21cd None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:08.583822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827eada-72bb-4c1b-bd72-7eb79fce21cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:08.607939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c827eada-72bb-4c1b-bd72-7eb79fce21cd None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:08.610457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c827eada-72bb-4c1b-bd72-7eb79fce21cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:08.727987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1041/2097] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:08.740494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeccb8-ffc3-4b08-bf3f-2ac729eed50f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:08.740819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeccb8-ffc3-4b08-bf3f-2ac729eed50f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:08.745521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeccb8-ffc3-4b08-bf3f-2ac729eed50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:08.777913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:08.787352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-88eeccb8-ffc3-4b08-bf3f-2ac729eed50f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:08.788732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-88eeccb8-ffc3-4b08-bf3f-2ac729eed50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:08.797791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1057/2098] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:08.802223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827eada-72bb-4c1b-bd72-7eb79fce21cd None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:08.813340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827eada-72bb-4c1b-bd72-7eb79fce21cd None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:08.820535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:08.821236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:08.821525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:08.821803 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:08.822620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:08.822930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:08.824567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:08.838545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1042/2099] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:49:08.843195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eeccb8-ffc3-4b08-bf3f-2ac729eed50f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:08.854367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eeccb8-ffc3-4b08-bf3f-2ac729eed50f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:08.859863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:08.860552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:08.860907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:08.861319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:08.862547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:08.864969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:08.966800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1058/2100] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:08 2022] POST /identity/v3/projects => generated 359 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:08.973035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:08.983608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5e51b-dc11-49f3-9e0e-44f672bfba7f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:08.995523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:09.043851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.044530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.044850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.045134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.045836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.047825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.080452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.082234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:09.095817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580499a-3879-4958-a810-b81774a2c850 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.100925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.101867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.102157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.102440 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] PATH_INFO: `/v3/projects/e412de340ed84b3688ab4fba22c40eba` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.103659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e412de340ed84b3688ab4fba22c40eba)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.117704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.213298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1043/2101] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:08 2022] POST /identity/v3/users => generated 280 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:09.217491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.230248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6ccfb-4146-4248-848f-a127d45674af admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.239502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.240200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.240492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.240851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.241662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.243218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.255489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1044/2102] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:49:09.259653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.270121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfca51-574e-4e39-80af-5677784b1ffe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.275506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.276176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.276462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.276856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.277590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.279328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.294842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1045/2103] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:49:09.298872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.312271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e412de340ed84b3688ab4fba22c40eba', 'request_id': 'req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:09.315906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2784f-72a7-42f0-88de-c94818904d4c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.319765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e412de340ed84b3688ab4fba22c40eba', 'request_id': 'req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:09.327253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e412de340ed84b3688ab4fba22c40eba', 'request_id': 'req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:09.388253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.389225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.389776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.389994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] PATH_INFO: `/v3/projects/bcc0d6b5d48344819882ba724935b8ad/users/54e1dbd22b0640ab9b908c76cbb7d5af/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.404404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0d6b5d48344819882ba724935b8ad, user_id=54e1dbd22b0640ab9b908c76cbb7d5af, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.407706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.447674 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] The token cache is being invalidate because project e412de340ed84b3688ab4fba22c40eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:09.448121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:09.448980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fbf19a1c-966b-4b2d-9f86-29d70ab3c9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:09.450777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1059/2104] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:09 2022] DELETE /identity/v3/projects/e412de340ed84b3688ab4fba22c40eba => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:09.457434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bfb41-df99-4d2c-b7af-a7292d7c2af4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.457872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bfb41-df99-4d2c-b7af-a7292d7c2af4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.458266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bfb41-df99-4d2c-b7af-a7292d7c2af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.488223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1e1bfb41-df99-4d2c-b7af-a7292d7c2af4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:09.488223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1e1bfb41-df99-4d2c-b7af-a7292d7c2af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:09.488223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1060/2105] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:09.495190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b921c37-2d20-40cf-99bb-2332450097b0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.495482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b921c37-2d20-40cf-99bb-2332450097b0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.495767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b921c37-2d20-40cf-99bb-2332450097b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.514829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1b921c37-2d20-40cf-99bb-2332450097b0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:09.516378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1b921c37-2d20-40cf-99bb-2332450097b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:09.524201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1061/2106] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:09.528381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b921c37-2d20-40cf-99bb-2332450097b0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.538280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b921c37-2d20-40cf-99bb-2332450097b0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.543001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8155e1a-712c-4f4b-aa53-7349cc23b8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.543637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8155e1a-712c-4f4b-aa53-7349cc23b8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.543926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8155e1a-712c-4f4b-aa53-7349cc23b8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.544207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8155e1a-712c-4f4b-aa53-7349cc23b8c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.544975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8155e1a-712c-4f4b-aa53-7349cc23b8c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:09.545289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8155e1a-712c-4f4b-aa53-7349cc23b8c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.546931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8155e1a-712c-4f4b-aa53-7349cc23b8c8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.556851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1046/2107] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:09 2022] PUT /identity/v3/projects/bcc0d6b5d48344819882ba724935b8ad/users/54e1dbd22b0640ab9b908c76cbb7d5af/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:09.562147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1062/2108] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:49:09.563184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.627179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.628070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.628525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.632585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14c64b-defd-467e-8308-15d3e2891446 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.638896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.639679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.640571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.641017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.642284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.644723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.665718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] MFA Rules not processed for user `54e1dbd22b0640ab9b908c76cbb7d5af`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:09.667427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:09.737104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1063/2109] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:09 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:09.767414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.768833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:09.785224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1047/2110] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:09 2022] POST /identity/v3/projects => generated 381 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:09.789552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.830296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e81c4-6566-47ac-8c7a-998fddb40e71 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.853468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804ea30-b5e5-4fb2-b01b-ed98ccd8202a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.858400 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.859059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.859345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.859625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.860316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.862273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.882380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.883038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.886287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:09.886287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:09.941427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:09.943352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:09.956798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1064/2111] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:09 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:49:09.975739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.977629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:09.992734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1048/2112] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:09 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:49:09.992985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec11f5-de9a-45de-bfbd-c5a6773bd2a0 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:09.993734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:09.998016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:09.998917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:09.999195 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.000140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] PATH_INFO: `/v3/users/54e1dbd22b0640ab9b908c76cbb7d5af` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.001454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e1dbd22b0640ab9b908c76cbb7d5af)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.003490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.004336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00e0e-3ae8-4cd5-9889-34295d368656 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:10.009603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:10.010267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.010552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.010827 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.011553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.013847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.041913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1049/2113] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:10.047589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:10.070795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1c7b-d30a-41e1-854a-98cc2f8d45ca admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:10.084127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:10.086392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.086790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.087173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.088071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.090281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.103753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1050/2114] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:10.114190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:10.127326 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658eca15-ea12-485d-b404-4567def578b7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:10.135898 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45378c-ef4d-444f-a08b-48a62e104b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:10.136916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45378c-ef4d-444f-a08b-48a62e104b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.137287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45378c-ef4d-444f-a08b-48a62e104b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.137634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45378c-ef4d-444f-a08b-48a62e104b53 admin admin] PATH_INFO: `/v3/projects/e8eb0355a49f41a39727c4dbd055523f/users/2fa8454dd1ee49af816fc2e119daac48/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.154033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45378c-ef4d-444f-a08b-48a62e104b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8eb0355a49f41a39727c4dbd055523f, user_id=2fa8454dd1ee49af816fc2e119daac48, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.156838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45378c-ef4d-444f-a08b-48a62e104b53 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.257866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e1dbd22b0640ab9b908c76cbb7d5af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:10.269865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e1dbd22b0640ab9b908c76cbb7d5af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:10.270617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e1dbd22b0640ab9b908c76cbb7d5af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:10.274779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1051/2115] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:10 2022] PUT /identity/v3/projects/e8eb0355a49f41a39727c4dbd055523f/users/2fa8454dd1ee49af816fc2e119daac48/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:10.278819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e1dbd22b0640ab9b908c76cbb7d5af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:10.279784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-37b06abd-b99a-42ff-bea1-643b966619e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e1dbd22b0640ab9b908c76cbb7d5af'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:10.285012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.285598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.285987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.314296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] MFA Rules not processed for user `2fa8454dd1ee49af816fc2e119daac48`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:10.315594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:10.363772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:10.373549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1065/2116] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:09 2022] DELETE /identity/v3/users/54e1dbd22b0640ab9b908c76cbb7d5af => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:10.378191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.378486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.378767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.400147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:10.401540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:10.419561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1052/2117] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:10.423782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:10.425491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:10.446126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:10.492523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1066/2118] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:10.509263 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:10.512584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:10.535614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1127f-05ac-44d4-88bf-01bba439869b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:10.543897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:10.543897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.543897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.543897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.544287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.545998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.590749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3508eb6-05ee-4e19-ad20-e883017110a9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:10.595840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:10.596783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.596783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.597077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.611170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.612686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.622530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1067/2119] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:10.737102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1053/2120] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:10 2022] POST /identity/v3/projects => generated 379 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:49:10.741340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:10.752280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f991f0-0cd8-4684-a197-bb71b5836625 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:10.815381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:10.816104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.816392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.816707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.817427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.819405 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.957350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1068/2121] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:10 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:10.962197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:10.975233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76290-5bfb-411e-b7bb-1df746f3ac74 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:10.980042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:10.980830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:10.981190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:10.981544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:10.982352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:10.984567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:10.995766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1054/2122] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:11.000979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:11.016116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e03e6d-b4a8-42ce-81e7-e8074635ccef admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:11.023418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:11.027595 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:11.027595 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:11.027595 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] PATH_INFO: `/v3/projects/ed2005cb54094b26b93cbeaaa77876ab/users/fbe38d8ff93c4f4ab44390c57f08bb4d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:11.045540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2005cb54094b26b93cbeaaa77876ab, user_id=fbe38d8ff93c4f4ab44390c57f08bb4d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:11.048819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:11.081043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:11.158015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:11.158764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1069/2123] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:10 2022] PUT /identity/v3/projects/ed2005cb54094b26b93cbeaaa77876ab/users/fbe38d8ff93c4f4ab44390c57f08bb4d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:11.162974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:11.176913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:11.198496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae94c6f-d970-4081-aa17-024a114a83b3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:11.273564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3be5a-7539-4958-8aa6-47830770dde4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:11.281982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:11.283100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:11.283553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:11.283838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:11.284889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:11.286550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:11.288517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:11.289503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:11.289855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:11.290204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] PATH_INFO: `/v3/projects/bcc0d6b5d48344819882ba724935b8ad` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:11.291506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc0d6b5d48344819882ba724935b8ad)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:11.293735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:11.297785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1070/2124] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:11.302381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:11.314028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a56ad-d97a-41e4-8f1d-9c8b052048ac admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:11.319107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e733cb9-cdba-462c-9bbd-ed89bddcdf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:11.320045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e733cb9-cdba-462c-9bbd-ed89bddcdf79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:11.320327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e733cb9-cdba-462c-9bbd-ed89bddcdf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:11.320632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e733cb9-cdba-462c-9bbd-ed89bddcdf79 admin admin] PATH_INFO: `/v3/projects/ed2005cb54094b26b93cbeaaa77876ab/users/fbe38d8ff93c4f4ab44390c57f08bb4d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:11.322843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e733cb9-cdba-462c-9bbd-ed89bddcdf79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2005cb54094b26b93cbeaaa77876ab, user_id=fbe38d8ff93c4f4ab44390c57f08bb4d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:11.325633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e733cb9-cdba-462c-9bbd-ed89bddcdf79 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:11.337552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1071/2125] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:11 2022] PUT /identity/v3/projects/ed2005cb54094b26b93cbeaaa77876ab/users/fbe38d8ff93c4f4ab44390c57f08bb4d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:11.340297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:11.361642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:11.361933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:11.362209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:11.391340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] MFA Rules not processed for user `fbe38d8ff93c4f4ab44390c57f08bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:11.392846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:11.469921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc0d6b5d48344819882ba724935b8ad', 'request_id': 'req-c0470d69-a4ed-48fe-82bf-1686ea3916eb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:11.476917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1072/2126] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:11.478378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc0d6b5d48344819882ba724935b8ad', 'request_id': 'req-c0470d69-a4ed-48fe-82bf-1686ea3916eb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:11.488343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:11.489838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:11.547266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe4aa6-5418-47d2-855f-c29ef900ac5e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:11.596557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb4da9-68fa-49e6-8d28-7826aa6eeb17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:11.597163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb4da9-68fa-49e6-8d28-7826aa6eeb17 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:11.597446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb4da9-68fa-49e6-8d28-7826aa6eeb17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:11.597722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb4da9-68fa-49e6-8d28-7826aa6eeb17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:11.623499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc0d6b5d48344819882ba724935b8ad', 'request_id': 'req-c0470d69-a4ed-48fe-82bf-1686ea3916eb'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:11.636039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] The token cache is being invalidate because project bcc0d6b5d48344819882ba724935b8ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:11.636428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:11.637195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:11.639255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1055/2127] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:11 2022] DELETE /identity/v3/projects/bcc0d6b5d48344819882ba724935b8ad => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:11.666918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb4da9-68fa-49e6-8d28-7826aa6eeb17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:11.668518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb4da9-68fa-49e6-8d28-7826aa6eeb17 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:11.681033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1073/2128] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:11.992673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:12.117506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:12.287218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:12.288688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:12.320699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:12.351956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0470d69-a4ed-48fe-82bf-1686ea3916eb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:12.356992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:12.357855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:12.358140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:12.358416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] PATH_INFO: `/v3/users/2fa8454dd1ee49af816fc2e119daac48` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:12.359737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa8454dd1ee49af816fc2e119daac48)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:12.361626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:12.567121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa8454dd1ee49af816fc2e119daac48'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:12.574780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa8454dd1ee49af816fc2e119daac48'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:12.677008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa8454dd1ee49af816fc2e119daac48'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:12.680258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa8454dd1ee49af816fc2e119daac48'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:12.680258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa8454dd1ee49af816fc2e119daac48'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:12.701716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1056/2129] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:12 2022] DELETE /identity/v3/users/2fa8454dd1ee49af816fc2e119daac48 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:12.706306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:12.706599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:12.706882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:12.723848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:12.725088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:12.783576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1074/2130] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:12.793286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:12.794716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:12.852904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c79e61-75ba-471c-a6dd-50c28d9cf62e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:12.856792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:12.857532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:12.857825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:12.858108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:12.871839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:12.873601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:12.887131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1057/2131] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:13.434291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:13.436070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:13.455634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173641a2-a89d-4fa4-a88f-b70f3422b119 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:13.462948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:13.466079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:13.466503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:13.466816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] PATH_INFO: `/v3/projects/e8eb0355a49f41a39727c4dbd055523f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:13.468141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8eb0355a49f41a39727c4dbd055523f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:13.470506 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:13.584108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8eb0355a49f41a39727c4dbd055523f', 'request_id': 'req-a8af10c0-eafd-413b-9bb6-83df448007d1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:13.590460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8eb0355a49f41a39727c4dbd055523f', 'request_id': 'req-a8af10c0-eafd-413b-9bb6-83df448007d1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:13.596812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8eb0355a49f41a39727c4dbd055523f', 'request_id': 'req-a8af10c0-eafd-413b-9bb6-83df448007d1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:13.670972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] The token cache is being invalidate because project e8eb0355a49f41a39727c4dbd055523f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:13.671264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:13.671899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:13.673531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1075/2132] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:13 2022] DELETE /identity/v3/projects/e8eb0355a49f41a39727c4dbd055523f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:13.677518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:13.687587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db7047-f66f-41e1-848a-96367c8f5e05 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:13.739660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:13.752513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:13.753548 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:13.753863 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:13.754146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] PATH_INFO: `/v3/users/fbe38d8ff93c4f4ab44390c57f08bb4d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:13.755980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbe38d8ff93c4f4ab44390c57f08bb4d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:13.758003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:13.994738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbe38d8ff93c4f4ab44390c57f08bb4d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.003034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbe38d8ff93c4f4ab44390c57f08bb4d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.168061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbe38d8ff93c4f4ab44390c57f08bb4d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.168856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbe38d8ff93c4f4ab44390c57f08bb4d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.169576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-0d0a72c5-078d-4ee3-9449-90312471db0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbe38d8ff93c4f4ab44390c57f08bb4d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.189862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1058/2133] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:13 2022] DELETE /identity/v3/users/fbe38d8ff93c4f4ab44390c57f08bb4d => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:14.656718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:14.659767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:14.673700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af10c0-eafd-413b-9bb6-83df448007d1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:14.683352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:14.734756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:14.735629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:14.735920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:14.736204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] PATH_INFO: `/v3/projects/ed2005cb54094b26b93cbeaaa77876ab` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:14.737439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2005cb54094b26b93cbeaaa77876ab)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:14.739424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:14.847122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2005cb54094b26b93cbeaaa77876ab', 'request_id': 'req-abf99a49-acf8-47e9-8df6-2b7cd1841495'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.853284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2005cb54094b26b93cbeaaa77876ab', 'request_id': 'req-abf99a49-acf8-47e9-8df6-2b7cd1841495'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.858749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2005cb54094b26b93cbeaaa77876ab', 'request_id': 'req-abf99a49-acf8-47e9-8df6-2b7cd1841495'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.976044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] The token cache is being invalidate because project ed2005cb54094b26b93cbeaaa77876ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:14.977350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.978053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-abf99a49-acf8-47e9-8df6-2b7cd1841495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:14.979642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1076/2134] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:14 2022] DELETE /identity/v3/projects/ed2005cb54094b26b93cbeaaa77876ab => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:14.986900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42296f0b-84d4-404c-8f95-43252c4f5b17 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:14.987999 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42296f0b-84d4-404c-8f95-43252c4f5b17 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:14.988412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42296f0b-84d4-404c-8f95-43252c4f5b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.010231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-42296f0b-84d4-404c-8f95-43252c4f5b17 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:15.011451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-42296f0b-84d4-404c-8f95-43252c4f5b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:15.076378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1059/2135] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:15.084274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57041f42-741a-4bf8-88a7-4f6ce817f039 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.084682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57041f42-741a-4bf8-88a7-4f6ce817f039 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.085029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57041f42-741a-4bf8-88a7-4f6ce817f039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.104643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-57041f42-741a-4bf8-88a7-4f6ce817f039 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:15.105806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-57041f42-741a-4bf8-88a7-4f6ce817f039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:15.115441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1077/2136] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:15.121098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42296f0b-84d4-404c-8f95-43252c4f5b17 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:15.134429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42296f0b-84d4-404c-8f95-43252c4f5b17 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:15.140236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:15.140986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.141285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.141562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.142555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:15.142871 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:15.144494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:15.158042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1060/2137] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:49:15.162412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57041f42-741a-4bf8-88a7-4f6ce817f039 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:15.173863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57041f42-741a-4bf8-88a7-4f6ce817f039 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:15.179451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:15.180112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.180430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.180739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.181450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:15.189695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:15.293913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1078/2138] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:15 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:49:15.295123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:15.306391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae60fc0-b733-4cee-9957-a4c2ec1f0a84 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:15.359811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:15.366947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:15.367634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.367925 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.368214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.368956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:15.371098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:15.513998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1061/2139] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:15 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:15.519647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:15.533065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2726e-8221-43a4-87c3-cdaf43710606 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:15.537971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:15.538653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.538944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.539227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.540061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:15.541729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:15.552501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1079/2140] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:15.556559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:15.567149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6f0d2-5c97-444c-9b74-adb2e83aa7cb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:15.571830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97dafaf-06d7-4c05-a3ec-ebc40d72431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:15.572535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dafaf-06d7-4c05-a3ec-ebc40d72431c admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.572853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dafaf-06d7-4c05-a3ec-ebc40d72431c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.584710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dafaf-06d7-4c05-a3ec-ebc40d72431c admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.584710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97dafaf-06d7-4c05-a3ec-ebc40d72431c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:15.584710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97dafaf-06d7-4c05-a3ec-ebc40d72431c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:15.587083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1062/2141] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:15.591347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:15.602163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90806783-59c5-4bad-8b18-c06f7232c709 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:15.606893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:15.607778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.608065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.608347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] PATH_INFO: `/v3/projects/a23e9d2676634c97a26554210617d7e0/users/d53d1c4d41144f92b26f6eddb5b01509/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.624015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23e9d2676634c97a26554210617d7e0, user_id=d53d1c4d41144f92b26f6eddb5b01509, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:15.626803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:15.732908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1080/2142] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:15 2022] PUT /identity/v3/projects/a23e9d2676634c97a26554210617d7e0/users/d53d1c4d41144f92b26f6eddb5b01509/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:15.743662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26304-e30b-434d-acdf-a9f8e6947b96 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.744026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26304-e30b-434d-acdf-a9f8e6947b96 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.744308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26304-e30b-434d-acdf-a9f8e6947b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.779228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1fd26304-e30b-434d-acdf-a9f8e6947b96 None None] MFA Rules not processed for user `d53d1c4d41144f92b26f6eddb5b01509`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:15.780723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1fd26304-e30b-434d-acdf-a9f8e6947b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:15.876361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1063/2143] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:15.884679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:15.928431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:15.948013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb1567-57e5-4674-84c9-80ad6e93ecdb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:15.952843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:15.953533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:15.954402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:15.954672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:15.955368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:15.957341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:16.061500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1081/2144] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:15 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:16.065676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26304-e30b-434d-acdf-a9f8e6947b96 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:16.077031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26304-e30b-434d-acdf-a9f8e6947b96 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:16.133514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:16.134157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:16.134441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:16.134717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:16.135412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:16.137409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:16.265682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1064/2145] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:16 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:49:16.270149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:16.280310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6bf1d-2261-4f00-bfad-f2f4e3ed8843 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:16.285682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:16.286335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:16.286620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:16.286896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:16.287663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:16.289274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:16.300183 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1082/2146] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:16.305942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:16.315986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f967f-f730-4b8d-b1c1-92c87a413b1d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:16.320573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:16.321474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:16.321762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:16.322041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] PATH_INFO: `/v3/projects/c2f38f7aa2c046868f2e5a23106297b0/users/3be87ea1fe0148cdb8540e9edd67bbb3/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:16.338560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f38f7aa2c046868f2e5a23106297b0, user_id=3be87ea1fe0148cdb8540e9edd67bbb3, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:16.341297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:16.434947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1065/2147] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:16 2022] PUT /identity/v3/projects/c2f38f7aa2c046868f2e5a23106297b0/users/3be87ea1fe0148cdb8540e9edd67bbb3/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:16.438223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:16.448973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79704-f61d-409b-8058-cf4e3ddf1250 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:16.515035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ccf94-e4c4-4453-9f0a-f6a9f2d5fbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:16.515718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ccf94-e4c4-4453-9f0a-f6a9f2d5fbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:16.516007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ccf94-e4c4-4453-9f0a-f6a9f2d5fbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:16.516903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ccf94-e4c4-4453-9f0a-f6a9f2d5fbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:16.517647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ccf94-e4c4-4453-9f0a-f6a9f2d5fbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:16.519269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ccf94-e4c4-4453-9f0a-f6a9f2d5fbb8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:16.530443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1083/2148] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:16.534934 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:16.545535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618d6bb-5a97-475c-9b8d-571c1ce14b50 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:16.551388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:16.552315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:16.552621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:16.552912 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] PATH_INFO: `/v3/projects/c2f38f7aa2c046868f2e5a23106297b0/users/3be87ea1fe0148cdb8540e9edd67bbb3/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:16.555409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f38f7aa2c046868f2e5a23106297b0, user_id=3be87ea1fe0148cdb8540e9edd67bbb3, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:16.557890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:16.568524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1066/2149] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:16 2022] PUT /identity/v3/projects/c2f38f7aa2c046868f2e5a23106297b0/users/3be87ea1fe0148cdb8540e9edd67bbb3/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:16.579846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbdfe3-3aa0-49cc-bcfd-b779333240b1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:16.580136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbdfe3-3aa0-49cc-bcfd-b779333240b1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:16.580411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbdfe3-3aa0-49cc-bcfd-b779333240b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:16.624956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-06fbdfe3-3aa0-49cc-bcfd-b779333240b1 None None] MFA Rules not processed for user `3be87ea1fe0148cdb8540e9edd67bbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:16.624956 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-06fbdfe3-3aa0-49cc-bcfd-b779333240b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:16.697760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1084/2150] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:16.710471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:16.712227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:16.735824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:16.773761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0e05-74a2-4d28-b4d1-3686a9222ed3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:16.778152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:16.779031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:16.779391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:16.779894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:16.798867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:16.800354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:16.814262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1067/2151] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:17.213125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbdfe3-3aa0-49cc-bcfd-b779333240b1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:17.225750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbdfe3-3aa0-49cc-bcfd-b779333240b1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:17.234706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37f47a-3f0e-4826-a6e0-196e3001e5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:17.236098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37f47a-3f0e-4826-a6e0-196e3001e5fc service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:17.236349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37f47a-3f0e-4826-a6e0-196e3001e5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:17.236736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37f47a-3f0e-4826-a6e0-196e3001e5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:17.272442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:17.298328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37f47a-3f0e-4826-a6e0-196e3001e5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:17.299779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37f47a-3f0e-4826-a6e0-196e3001e5fc service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:17.308936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1085/2152] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:18.264254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:18.331736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faaf26e-b71c-4c08-b4f0-1886a7359175 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:18.336488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:18.337391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:18.337682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:18.337967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] PATH_INFO: `/v3/users/d53d1c4d41144f92b26f6eddb5b01509` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:18.339164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53d1c4d41144f92b26f6eddb5b01509)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:18.341166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:18.617290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53d1c4d41144f92b26f6eddb5b01509'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:18.625014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53d1c4d41144f92b26f6eddb5b01509'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:18.753464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53d1c4d41144f92b26f6eddb5b01509'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:18.754698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53d1c4d41144f92b26f6eddb5b01509'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:18.755878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53d1c4d41144f92b26f6eddb5b01509'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:18.778207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1068/2153] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:18 2022] DELETE /identity/v3/users/d53d1c4d41144f92b26f6eddb5b01509 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:18.783529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:18.784012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:18.784127 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:18.801732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:18.802933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:18.894207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1086/2154] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:18.904450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:18.906169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:18.952917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:18.966448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775f330-0dc4-4419-914e-6abdfbf9fd7f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:18.970860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:18.971756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:18.972096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:18.972380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:18.981788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:18.983242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:19.004682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1069/2155] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:19.516668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:19.518117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:19.548243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c7ef7-066f-493c-89d8-741501a370bb None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:19.553544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:19.554757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:19.555287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:19.555726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] PATH_INFO: `/v3/projects/a23e9d2676634c97a26554210617d7e0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:19.557122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23e9d2676634c97a26554210617d7e0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:19.559682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:19.767425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23e9d2676634c97a26554210617d7e0', 'request_id': 'req-f57204d8-7ca2-431a-b409-450e5310a6e9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:19.775645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23e9d2676634c97a26554210617d7e0', 'request_id': 'req-f57204d8-7ca2-431a-b409-450e5310a6e9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:19.782532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23e9d2676634c97a26554210617d7e0', 'request_id': 'req-f57204d8-7ca2-431a-b409-450e5310a6e9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:19.918611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] The token cache is being invalidate because project a23e9d2676634c97a26554210617d7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:19.919259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:19.919854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f57204d8-7ca2-431a-b409-450e5310a6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:19.925990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1087/2156] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:19 2022] DELETE /identity/v3/projects/a23e9d2676634c97a26554210617d7e0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:19.927177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:19.937234 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4277d83-2c79-49b3-a321-a66c17458227 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:19.994081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:19.995078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:19.995379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:19.995648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] PATH_INFO: `/v3/users/3be87ea1fe0148cdb8540e9edd67bbb3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:19.996887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be87ea1fe0148cdb8540e9edd67bbb3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:19.998835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:20.253303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:20.253601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:20.253953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:20.275388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] MFA Rules not processed for user `8873b1c9f58a4adea93ebe21f5f3263d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:20.276653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:20.300102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be87ea1fe0148cdb8540e9edd67bbb3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:20.309164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be87ea1fe0148cdb8540e9edd67bbb3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:20.347787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1088/2157] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:20.448011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be87ea1fe0148cdb8540e9edd67bbb3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:20.448678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be87ea1fe0148cdb8540e9edd67bbb3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:20.449312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fba772c6-f108-41fb-894c-ec0371856344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be87ea1fe0148cdb8540e9edd67bbb3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:20.467631 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1070/2158] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:19 2022] DELETE /identity/v3/users/3be87ea1fe0148cdb8540e9edd67bbb3 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:20.927287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:20.929802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:20.940241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:20.947961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3f2dc-cb57-4558-97a3-56a5e868f4ea None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:21.012082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:21.012911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.013202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.013485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] PATH_INFO: `/v3/projects/c2f38f7aa2c046868f2e5a23106297b0` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.014676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f38f7aa2c046868f2e5a23106297b0)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:21.016710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:21.210187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f38f7aa2c046868f2e5a23106297b0', 'request_id': 'req-6634eef2-b588-42bc-a564-1449cd534cd2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:21.224332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f38f7aa2c046868f2e5a23106297b0', 'request_id': 'req-6634eef2-b588-42bc-a564-1449cd534cd2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:21.229765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f38f7aa2c046868f2e5a23106297b0', 'request_id': 'req-6634eef2-b588-42bc-a564-1449cd534cd2'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:21.368355 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] The token cache is being invalidate because project c2f38f7aa2c046868f2e5a23106297b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:21.368675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:21.369408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6634eef2-b588-42bc-a564-1449cd534cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:21.371161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1089/2159] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:20 2022] DELETE /identity/v3/projects/c2f38f7aa2c046868f2e5a23106297b0 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:21.376007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e227a8-7e81-438e-9d04-177562ba3c84 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.376468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e227a8-7e81-438e-9d04-177562ba3c84 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.384478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e227a8-7e81-438e-9d04-177562ba3c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.395570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-76e227a8-7e81-438e-9d04-177562ba3c84 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:21.396895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-76e227a8-7e81-438e-9d04-177562ba3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:21.456559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1071/2160] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:21.464377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72cc0-4cf1-4a7a-86db-d50b4d5674d4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.464702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72cc0-4cf1-4a7a-86db-d50b4d5674d4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.464987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72cc0-4cf1-4a7a-86db-d50b4d5674d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.485390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-12a72cc0-4cf1-4a7a-86db-d50b4d5674d4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:21.486792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-12a72cc0-4cf1-4a7a-86db-d50b4d5674d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:21.495004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1090/2161] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:21.499465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e227a8-7e81-438e-9d04-177562ba3c84 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:21.506059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:21.512837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e227a8-7e81-438e-9d04-177562ba3c84 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:21.518683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:21.519550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.519839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.520115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.520882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:21.521189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:21.522845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:21.536629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1072/2162] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:49:21.542153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a72cc0-4cf1-4a7a-86db-d50b4d5674d4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:21.552234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a72cc0-4cf1-4a7a-86db-d50b4d5674d4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:21.557079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:21.557734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.558055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.559242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.559242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:21.561111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:21.734049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1091/2163] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:21 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:21.738865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:21.748974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb47f5-a41a-473a-a95c-789ae29871ee admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:21.807820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:21.808443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.808772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.809064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.809772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:21.811710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:21.923121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1073/2164] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:21 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:49:21.927290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:21.937819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8885-12f2-4191-887f-3daa79d7b99d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:21.943050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:21.943760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.944046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.944327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.945099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:21.946706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:21.958010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1092/2165] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:21.962067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:21.972012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896e28-9a5c-4e34-a585-2af35d060804 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:21.976585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909d31f-6cad-4f39-8b49-03ddf992f08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:21.977270 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d31f-6cad-4f39-8b49-03ddf992f08d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:21.977557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d31f-6cad-4f39-8b49-03ddf992f08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:21.977833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909d31f-6cad-4f39-8b49-03ddf992f08d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:21.978562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909d31f-6cad-4f39-8b49-03ddf992f08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:21.980180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909d31f-6cad-4f39-8b49-03ddf992f08d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:21.990019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1074/2166] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:22.001654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:22.012131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a6d78-5dcd-45db-8f83-913592f7481f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:22.017872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:22.018778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.019036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.019310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] PATH_INFO: `/v3/projects/2bbb35c59b8041e3bf54bca963f9300d/users/bb9c98944c944b4b8fb5e5ae79123947/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.035855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bbb35c59b8041e3bf54bca963f9300d, user_id=bb9c98944c944b4b8fb5e5ae79123947, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:22.039004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:22.172879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1093/2167] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:22 2022] PUT /identity/v3/projects/2bbb35c59b8041e3bf54bca963f9300d/users/bb9c98944c944b4b8fb5e5ae79123947/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:22.184242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dac74-b6e6-49ca-bb75-6e27aa9e5b18 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.184538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dac74-b6e6-49ca-bb75-6e27aa9e5b18 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.184855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dac74-b6e6-49ca-bb75-6e27aa9e5b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.211818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-750dac74-b6e6-49ca-bb75-6e27aa9e5b18 None None] MFA Rules not processed for user `bb9c98944c944b4b8fb5e5ae79123947`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:22.213093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-750dac74-b6e6-49ca-bb75-6e27aa9e5b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:22.279089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1075/2168] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:22.283552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:22.316387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:22.347535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b6843-8eb2-41e6-8009-7b2e3048e022 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:22.352382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:22.353083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.353374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.353649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.354381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:22.356326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:22.506012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1094/2169] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:22 2022] POST /identity/v3/projects => generated 383 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:22.510130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dac74-b6e6-49ca-bb75-6e27aa9e5b18 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:22.521468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dac74-b6e6-49ca-bb75-6e27aa9e5b18 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:22.574205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:22.580183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:22.580904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.581193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.581474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.582183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:22.584116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:22.729935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1076/2170] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:22 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:22.731394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:22.742634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4da6-e990-44b1-a619-671063d4098c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:22.747557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:22.748237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.748544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.748865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.749603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:22.751222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:22.762646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1095/2171] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:22.766746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:22.776740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7dc9-f93b-498a-9762-2465544e609e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:22.781336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67bcb-195f-4a86-8618-c1ce4da147a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:22.782585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67bcb-195f-4a86-8618-c1ce4da147a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.782867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67bcb-195f-4a86-8618-c1ce4da147a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.783153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67bcb-195f-4a86-8618-c1ce4da147a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.783880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67bcb-195f-4a86-8618-c1ce4da147a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:22.785538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67bcb-195f-4a86-8618-c1ce4da147a0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:22.795744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1077/2172] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:22.800810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:22.810992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ce4de-2a6f-4ba3-8d92-fc0f65237487 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:22.816357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:22.817270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.817552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.817890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] PATH_INFO: `/v3/projects/171b148d457d471ca86844a25fbe74de/users/ed9679c96c514b638b2587123e89788b/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.833233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=171b148d457d471ca86844a25fbe74de, user_id=ed9679c96c514b638b2587123e89788b, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:22.835926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:22.957313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1096/2173] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:22 2022] PUT /identity/v3/projects/171b148d457d471ca86844a25fbe74de/users/ed9679c96c514b638b2587123e89788b/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:22.967742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6e8d-7ab2-4d93-9fe9-d9cba2708cf1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:22.968040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6e8d-7ab2-4d93-9fe9-d9cba2708cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:22.968368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6e8d-7ab2-4d93-9fe9-d9cba2708cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:22.995680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-80ad6e8d-7ab2-4d93-9fe9-d9cba2708cf1 None None] MFA Rules not processed for user `ed9679c96c514b638b2587123e89788b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:22.996991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-80ad6e8d-7ab2-4d93-9fe9-d9cba2708cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:23.061634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1078/2174] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:23.066857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:23.136206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd880e-fc38-422a-8e04-6f511bf3a7a8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:23.141715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:23.142386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.142673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.142950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.143646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:23.145667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:23.265735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1097/2175] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:23 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:49:23.269246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6e8d-7ab2-4d93-9fe9-d9cba2708cf1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:23.279769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6e8d-7ab2-4d93-9fe9-d9cba2708cf1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:23.346110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:23.346791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.347078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.347359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.348058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:23.350062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:23.445301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1079/2176] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:23 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:23.446525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:23.457427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce9d28-e175-4b06-9da1-d96f5f30fb03 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:23.462270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:23.463007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.463288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.463561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.464410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:23.466072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:23.480419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1098/2177] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:23.485817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:23.496184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa419826-8fe5-46e2-b6ab-7c17bfb00fe6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:23.501222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:23.502189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.502536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.502874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] PATH_INFO: `/v3/projects/ee7ac9ab55514dbea4f8246e43971d2a/users/d15341f2072e4e1aad1f8a0a30dd18ff/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.518372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7ac9ab55514dbea4f8246e43971d2a, user_id=d15341f2072e4e1aad1f8a0a30dd18ff, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:23.521210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:23.597622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1080/2178] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:23 2022] PUT /identity/v3/projects/ee7ac9ab55514dbea4f8246e43971d2a/users/d15341f2072e4e1aad1f8a0a30dd18ff/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:23.601523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:23.612231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a6257-35b6-4d95-822a-590f6df006e5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:23.660682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:23.669476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd2ebd-96fd-4db0-bdfe-cbde7061d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:23.670136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd2ebd-96fd-4db0-bdfe-cbde7061d3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.670417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd2ebd-96fd-4db0-bdfe-cbde7061d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.670691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd2ebd-96fd-4db0-bdfe-cbde7061d3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.671414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd2ebd-96fd-4db0-bdfe-cbde7061d3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:23.673131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd2ebd-96fd-4db0-bdfe-cbde7061d3e8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:23.682743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1099/2179] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:49:23.687209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:23.697897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa68554-0a3d-4227-b6f9-475bddff179c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:23.703043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:23.704442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.704767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.705051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] PATH_INFO: `/v3/projects/ee7ac9ab55514dbea4f8246e43971d2a/users/d15341f2072e4e1aad1f8a0a30dd18ff/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.707067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7ac9ab55514dbea4f8246e43971d2a, user_id=d15341f2072e4e1aad1f8a0a30dd18ff, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:23.709836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:23.721131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1081/2180] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:23 2022] PUT /identity/v3/projects/ee7ac9ab55514dbea4f8246e43971d2a/users/d15341f2072e4e1aad1f8a0a30dd18ff/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:23.731451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e434-2e5a-44fb-b67a-968d2241d7dc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.731747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e434-2e5a-44fb-b67a-968d2241d7dc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.732028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7e434-2e5a-44fb-b67a-968d2241d7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.758794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-98e7e434-2e5a-44fb-b67a-968d2241d7dc None None] MFA Rules not processed for user `d15341f2072e4e1aad1f8a0a30dd18ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:23.760048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-98e7e434-2e5a-44fb-b67a-968d2241d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:23.830547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1100/2181] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:23.841580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:23.843397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:23.865375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:23.907717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37e4ae-025b-471e-be98-5d947016c98d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:23.912489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc928c1a-b361-4e27-bbc6-9ab9755213a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:23.913526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928c1a-b361-4e27-bbc6-9ab9755213a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:23.914168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928c1a-b361-4e27-bbc6-9ab9755213a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:23.914770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928c1a-b361-4e27-bbc6-9ab9755213a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:23.974518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc928c1a-b361-4e27-bbc6-9ab9755213a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:23.976027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc928c1a-b361-4e27-bbc6-9ab9755213a5 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:23.987667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1082/2182] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:26.491533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e434-2e5a-44fb-b67a-968d2241d7dc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:26.501729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7e434-2e5a-44fb-b67a-968d2241d7dc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:26.505578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:26.506225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:26.506515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:26.506789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:26.564839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:26.566279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:26.575613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1101/2183] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:27.170981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d4c4c-6474-40d4-b20a-955315de5efa None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:27.171860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d4c4c-6474-40d4-b20a-955315de5efa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:27.172177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d4c4c-6474-40d4-b20a-955315de5efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:27.190634 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9a5d4c4c-6474-40d4-b20a-955315de5efa None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:27.191853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9a5d4c4c-6474-40d4-b20a-955315de5efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:27.251102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1083/2184] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:49:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:27.302188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:27.303844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:27.381619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283af76-4d19-4ed1-a791-199c0bf67ea7 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:27.387102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcda8b-516f-4505-8785-8a8f334aefa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:27.387761 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcda8b-516f-4505-8785-8a8f334aefa6 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:27.388046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcda8b-516f-4505-8785-8a8f334aefa6 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:27.388324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcda8b-516f-4505-8785-8a8f334aefa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:27.398051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcda8b-516f-4505-8785-8a8f334aefa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:27.399662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcda8b-516f-4505-8785-8a8f334aefa6 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:27.409380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1102/2185] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:49:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:49:29.058971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d4c4c-6474-40d4-b20a-955315de5efa None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:29.079738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d4c4c-6474-40d4-b20a-955315de5efa None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:29.093331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:29.099382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:29.101588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:29.102680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:29.128980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:29.131384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:29.160129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1084/2186] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:32.499046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452eeb2-c34d-4548-b3b3-38addc36ab3a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:32.499832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452eeb2-c34d-4548-b3b3-38addc36ab3a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:32.500513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452eeb2-c34d-4548-b3b3-38addc36ab3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:32.509353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:32.526911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b452eeb2-c34d-4548-b3b3-38addc36ab3a None None] MFA Rules not processed for user `34e7019b46d743a1b6b0d3315e69053f`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:32.528295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b452eeb2-c34d-4548-b3b3-38addc36ab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:32.601491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1103/2187] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:33.317653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:33.319783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:33.416260 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0462a7-d387-44a8-8a93-a7c4e82575ae service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:33.421596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:33.422743 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:33.423382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:33.423993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] PATH_INFO: `/v3/users/bef49100132c4ee1928d5463de2be0a9` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:33.425691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef49100132c4ee1928d5463de2be0a9)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:33.427926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:33.434600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:33.654202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef49100132c4ee1928d5463de2be0a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:33.664330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef49100132c4ee1928d5463de2be0a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:33.743635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef49100132c4ee1928d5463de2be0a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:33.744317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef49100132c4ee1928d5463de2be0a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:33.744964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef49100132c4ee1928d5463de2be0a9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:33.770445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1085/2188] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:33 2022] DELETE /identity/v3/users/bef49100132c4ee1928d5463de2be0a9 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:33.774814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:33.775283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:33.775372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:33.798582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:33.798985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:33.924935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1104/2189] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:33.935702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:33.937213 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:34.038654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742ba4e-6b39-4f7b-9103-91fd6e6ff06e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:34.042549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a9bd8-4815-4ae3-8943-cfd255961f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:34.043206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9bd8-4815-4ae3-8943-cfd255961f01 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:34.043491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9bd8-4815-4ae3-8943-cfd255961f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:34.043769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9bd8-4815-4ae3-8943-cfd255961f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:34.061485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a9bd8-4815-4ae3-8943-cfd255961f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:34.062892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a9bd8-4815-4ae3-8943-cfd255961f01 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:34.077498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1086/2190] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:35.009934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:35.009934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:35.050563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11c148-53d8-4be6-85a0-2a9261376653 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:35.059603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:35.060589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:35.061605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:35.061904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] PATH_INFO: `/v3/projects/18b35ba7e32c4774a4b0eb7889e941c4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:35.064367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b35ba7e32c4774a4b0eb7889e941c4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:35.077180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:35.264252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b35ba7e32c4774a4b0eb7889e941c4', 'request_id': 'req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.276118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b35ba7e32c4774a4b0eb7889e941c4', 'request_id': 'req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.284912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b35ba7e32c4774a4b0eb7889e941c4', 'request_id': 'req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.394677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] The token cache is being invalidate because project 18b35ba7e32c4774a4b0eb7889e941c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:35.395137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.396670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.404034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1105/2191] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:35 2022] DELETE /identity/v3/projects/18b35ba7e32c4774a4b0eb7889e941c4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:35.405626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0cd3c-0b74-49a2-8aaf-341bff78d43d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:35.408388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0cd3c-0b74-49a2-8aaf-341bff78d43d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:35.408696 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0cd3c-0b74-49a2-8aaf-341bff78d43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:35.440708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-13f0cd3c-0b74-49a2-8aaf-341bff78d43d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:35.442238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-13f0cd3c-0b74-49a2-8aaf-341bff78d43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:35.586742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:35.588934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:35.600615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1087/2192] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:35.606438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73c675-f026-4dd3-9b4b-b28aab5e3aec None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:35.606756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73c675-f026-4dd3-9b4b-b28aab5e3aec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:35.607069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73c675-f026-4dd3-9b4b-b28aab5e3aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:35.614795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e44aa9-565a-40d8-b3dc-e4e6a2236b24 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:35.628386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:35.630205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:35.630497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:35.630775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] PATH_INFO: `/v3/users/e9157a1696e043369b0de1e494a32976` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:35.632592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9157a1696e043369b0de1e494a32976)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:35.635626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:35.651155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4d73c675-f026-4dd3-9b4b-b28aab5e3aec None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:35.654294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4d73c675-f026-4dd3-9b4b-b28aab5e3aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:35.684175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1088/2193] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:35.688639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73c675-f026-4dd3-9b4b-b28aab5e3aec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:35.704351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73c675-f026-4dd3-9b4b-b28aab5e3aec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:35.711711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:35.712466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:35.712825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:35.713747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:35.714964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:35.715453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:35.717980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:35.739509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1089/2194] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:49:35.784798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:35.800520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14752a3d-940d-4443-98a5-53f530a720de admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:35.807905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:35.829784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:35.830469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:35.830759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:35.831888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:35.831888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:35.839245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:35.848882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:35.976894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9157a1696e043369b0de1e494a32976'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.987661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9157a1696e043369b0de1e494a32976'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.988757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9157a1696e043369b0de1e494a32976'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.995685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9157a1696e043369b0de1e494a32976'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:35.996418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9157a1696e043369b0de1e494a32976'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:36.420860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1090/2195] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:35 2022] POST /identity/v3/projects => generated 371 bytes in 644 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:36.420860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:36.432344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:36.499315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da0c9d-6024-48dc-8713-8d13eb695e1e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:36.503716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:36.504970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:36.505302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:36.505581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] PATH_INFO: `/v3/users/bb9c98944c944b4b8fb5e5ae79123947` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:36.506828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9c98944c944b4b8fb5e5ae79123947)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:36.508763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:36.574422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1106/2196] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:35 2022] DELETE /identity/v3/users/e9157a1696e043369b0de1e494a32976 => generated 0 bytes in 988 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:36.578495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:36.578495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:36.655791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359acf7-a6fc-4626-8f7f-6d0fdd934d3c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:36.671029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59d08e-0d07-471c-9394-541de5fdd612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:36.671666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d08e-0d07-471c-9394-541de5fdd612 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:36.672490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d08e-0d07-471c-9394-541de5fdd612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:36.672801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d08e-0d07-471c-9394-541de5fdd612 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:36.674227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59d08e-0d07-471c-9394-541de5fdd612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:36.677106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59d08e-0d07-471c-9394-541de5fdd612 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:36.895652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9c98944c944b4b8fb5e5ae79123947'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:36.896308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1107/2197] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:36 2022] POST /identity/v3/users => generated 286 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:36.898853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadaf4f-d423-481e-83fa-f4189c740a10 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:36.899364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadaf4f-d423-481e-83fa-f4189c740a10 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:36.902228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadaf4f-d423-481e-83fa-f4189c740a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:36.911808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9c98944c944b4b8fb5e5ae79123947'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:36.939341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2aadaf4f-d423-481e-83fa-f4189c740a10 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:36.943142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2aadaf4f-d423-481e-83fa-f4189c740a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:36.962713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1108/2198] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:36.962713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadaf4f-d423-481e-83fa-f4189c740a10 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:36.982167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadaf4f-d423-481e-83fa-f4189c740a10 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:36.985313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9c98944c944b4b8fb5e5ae79123947'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:36.985876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9c98944c944b4b8fb5e5ae79123947'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:36.986493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9c98944c944b4b8fb5e5ae79123947'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:36.989517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:36.990291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:36.990676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:36.991053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:36.991891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:36.993688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:37.017622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1109/2199] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:37.018642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:37.032960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1091/2200] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:36 2022] DELETE /identity/v3/users/bb9c98944c944b4b8fb5e5ae79123947 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:37.034611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:37.036114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:37.038764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:37.258245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745f820-16a2-41e1-8be0-cd3d668b26ac admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:37.289628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902bf41-1b08-45e3-8c6b-331bbcb40d30 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:37.304225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e67a62-8220-4af1-a069-5e68abf970df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:37.310401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e67a62-8220-4af1-a069-5e68abf970df admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:37.310778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e67a62-8220-4af1-a069-5e68abf970df admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:37.311072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e67a62-8220-4af1-a069-5e68abf970df admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:37.314647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e67a62-8220-4af1-a069-5e68abf970df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:37.317960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e67a62-8220-4af1-a069-5e68abf970df admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:37.330067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:37.343458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1092/2201] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:37.351674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:37.351854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:37.352083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:37.359039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:37.361498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:37.362493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:37.362973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:37.365930 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:37.380192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:37.382698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:37.397537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:37.398867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:37.400799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1110/2202] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:37.402342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:37.411913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:37.420249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1093/2203] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:37.440711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:37.442455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140a463-f11b-4e33-996b-ffd9f060eb1e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:37.449493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:37.450504 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b0383-bdee-499e-b78b-c132ce2e7172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:37.451670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0383-bdee-499e-b78b-c132ce2e7172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:37.452075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0383-bdee-499e-b78b-c132ce2e7172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:37.452412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0383-bdee-499e-b78b-c132ce2e7172 admin admin] PATH_INFO: `/v3/projects/ba6c36205c23430aa0de7e8b90e91224/users/a5fc663eb6fe425da20252c99e079d5b/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:37.472522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b0383-bdee-499e-b78b-c132ce2e7172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6c36205c23430aa0de7e8b90e91224, user_id=a5fc663eb6fe425da20252c99e079d5b, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:37.476069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b0383-bdee-499e-b78b-c132ce2e7172 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:37.490455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02ce6-caa8-4cb3-aacc-8356e4556f75 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:37.503425 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:37.504570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:37.505119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:37.505628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:37.534877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:37.536737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:37.552747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1111/2204] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:37 2022] PUT /identity/v3/projects/ba6c36205c23430aa0de7e8b90e91224/users/a5fc663eb6fe425da20252c99e079d5b/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:37.561248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:37.561583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:37.561826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:37.572998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1094/2205] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:49:37.612082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] MFA Rules not processed for user `a5fc663eb6fe425da20252c99e079d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:37.613582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:37.757772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1112/2206] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:37.774904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:37.777015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:37.862657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c25b6-2f69-4d15-abd1-394ed63e0a41 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:37.926839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:37.927566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:37.927910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:37.928203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:37.939608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:37.941234 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:37.956716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1095/2207] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:38.188765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:38.191513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:38.426657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57516a53-b240-4de3-b118-33d38231bb2d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:38.440451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:38.441378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:38.441665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:38.441947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] PATH_INFO: `/v3/projects/d551da8f382e4c41aa4892edd084436a` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:38.458330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d551da8f382e4c41aa4892edd084436a)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:38.466056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:38.720589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:38.736372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd551da8f382e4c41aa4892edd084436a', 'request_id': 'req-d53288ed-16ef-40e9-9da5-78272d6d5cee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:38.747135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd551da8f382e4c41aa4892edd084436a', 'request_id': 'req-d53288ed-16ef-40e9-9da5-78272d6d5cee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:38.756265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd551da8f382e4c41aa4892edd084436a', 'request_id': 'req-d53288ed-16ef-40e9-9da5-78272d6d5cee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:38.852235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] The token cache is being invalidate because project d551da8f382e4c41aa4892edd084436a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:38.852235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:38.852235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-d53288ed-16ef-40e9-9da5-78272d6d5cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:38.852515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:38.852952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1113/2208] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:38 2022] DELETE /identity/v3/projects/d551da8f382e4c41aa4892edd084436a => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:38.853945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:38.857452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64618b1b-0154-41d2-9c38-5d1c35313559 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:38.857804 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64618b1b-0154-41d2-9c38-5d1c35313559 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:38.858091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64618b1b-0154-41d2-9c38-5d1c35313559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:38.878544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e681b3c-0417-4a80-b5f3-93f9a3c1f53c service glance] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:38.900737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-64618b1b-0154-41d2-9c38-5d1c35313559 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:38.902670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-64618b1b-0154-41d2-9c38-5d1c35313559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:39.014715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:39.035894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.035894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.035894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] PATH_INFO: `/v3/projects/2bbb35c59b8041e3bf54bca963f9300d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.048648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bbb35c59b8041e3bf54bca963f9300d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:39.058559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:39.114100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1114/2209] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:39.115776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adcc00-4dad-4208-8170-61ee8d546b10 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.116181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adcc00-4dad-4208-8170-61ee8d546b10 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.116654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adcc00-4dad-4208-8170-61ee8d546b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.162679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-65adcc00-4dad-4208-8170-61ee8d546b10 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:39.167242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-65adcc00-4dad-4208-8170-61ee8d546b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:39.195824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1115/2210] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:39.197038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adcc00-4dad-4208-8170-61ee8d546b10 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:39.212478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adcc00-4dad-4208-8170-61ee8d546b10 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:39.233681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:39.234594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.234988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.235406 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.236340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:39.236787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:39.240350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:39.275723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:39.282055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1116/2211] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:49:39.283129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:39.295448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d680738-6eed-42d2-9d4f-02d2e726d011 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:39.300038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bbb35c59b8041e3bf54bca963f9300d', 'request_id': 'req-da7ba65a-9c7a-432f-a227-16f74bd2a93a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:39.324478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bbb35c59b8041e3bf54bca963f9300d', 'request_id': 'req-da7ba65a-9c7a-432f-a227-16f74bd2a93a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:39.425088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:39.426553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.427078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.427817 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.428831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:39.432752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:39.459496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bbb35c59b8041e3bf54bca963f9300d', 'request_id': 'req-da7ba65a-9c7a-432f-a227-16f74bd2a93a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:39.474039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] The token cache is being invalidate because project 2bbb35c59b8041e3bf54bca963f9300d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:39.474340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:39.478665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:39.496225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1096/2212] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:38 2022] DELETE /identity/v3/projects/2bbb35c59b8041e3bf54bca963f9300d => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:39.496225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:39.496225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:39.522187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ba65a-9c7a-432f-a227-16f74bd2a93a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:39.534872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:39.535820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.536117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.536407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] PATH_INFO: `/v3/users/ed9679c96c514b638b2587123e89788b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.537980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1117/2213] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:39 2022] POST /identity/v3/projects => generated 383 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:39.538322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9679c96c514b638b2587123e89788b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:39.540664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:39.543137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:39.548214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:39.669975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485ec92-fc2f-490b-8b68-5574425f7102 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:39.676010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:39.676882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.677285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.677663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.678693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:39.680904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:39.835445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1118/2214] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:39 2022] POST /identity/v3/users => generated 292 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:39.840679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:39.864474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9679c96c514b638b2587123e89788b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:39.868155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339985a3-0d24-4e2c-9f5a-f11a4e7d23a5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:39.877556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9679c96c514b638b2587123e89788b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:39.882249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:39.883372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.883818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.884252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.885330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:39.890318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:39.917565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1119/2215] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:39.921676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:39.934643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1447b24-8c73-4cfd-9877-2e3d08705c28 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:39.939657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:39.940350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:39.940696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:39.940987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:39.942311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:39.944167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:39.962356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1120/2216] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:39.967527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:39.990599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e7086-8956-41e8-af85-bbddc760bef9 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:39.999244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:40.000792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:40.001086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:40.002016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] PATH_INFO: `/v3/projects/d6aa269b9af048c1860128aa8db2bf28/users/611f51a2f6434e0ab78559d8b03493df/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:40.006870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9679c96c514b638b2587123e89788b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:40.008670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9679c96c514b638b2587123e89788b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:40.010152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-9136c068-e932-48ee-8be3-4d52281382b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9679c96c514b638b2587123e89788b'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:40.027947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6aa269b9af048c1860128aa8db2bf28, user_id=611f51a2f6434e0ab78559d8b03493df, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:40.031566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:40.054396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1097/2217] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:39 2022] DELETE /identity/v3/users/ed9679c96c514b638b2587123e89788b => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:40.221221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1121/2218] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:39 2022] PUT /identity/v3/projects/d6aa269b9af048c1860128aa8db2bf28/users/611f51a2f6434e0ab78559d8b03493df/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:40.226487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:40.226779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:40.227056 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:40.253010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:40.254667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:40.414961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:40.417050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1098/2219] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:40.430108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:40.431873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:40.583065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2b3af-9454-4e3c-923e-22303fc3eb0b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:40.587878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:40.589104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:40.590103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:40.590103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:40.658066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:40.692840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:40.735411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1122/2220] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:40.823519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:40.825252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:40.852943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b248d6-fe52-4c04-a468-df7c26c1cf50 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:40.868836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:40.873000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:40.873295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:40.873587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] PATH_INFO: `/v3/projects/171b148d457d471ca86844a25fbe74de` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:40.877255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171b148d457d471ca86844a25fbe74de)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:40.880356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:41.055796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:41.094712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171b148d457d471ca86844a25fbe74de', 'request_id': 'req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:41.121238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171b148d457d471ca86844a25fbe74de', 'request_id': 'req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:41.227065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171b148d457d471ca86844a25fbe74de', 'request_id': 'req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:41.256247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] The token cache is being invalidate because project 171b148d457d471ca86844a25fbe74de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:41.256968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:41.282669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:41.292022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1099/2221] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:40 2022] DELETE /identity/v3/projects/171b148d457d471ca86844a25fbe74de => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:41.292944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:41.326670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566dff6-c2be-470a-82c1-6bb019aa35af service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:41.574394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:41.575331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:41.575803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:41.576402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] PATH_INFO: `/v3/users/d15341f2072e4e1aad1f8a0a30dd18ff` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:41.578224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15341f2072e4e1aad1f8a0a30dd18ff)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:41.580374 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:41.615656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:42.436479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15341f2072e4e1aad1f8a0a30dd18ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:42.474281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15341f2072e4e1aad1f8a0a30dd18ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:42.475129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15341f2072e4e1aad1f8a0a30dd18ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:42.485003 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15341f2072e4e1aad1f8a0a30dd18ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:42.494006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1bf2a56e-ab0d-4c6d-8990-72067893f8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15341f2072e4e1aad1f8a0a30dd18ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:42.655634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1123/2222] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:41 2022] DELETE /identity/v3/users/d15341f2072e4e1aad1f8a0a30dd18ff => generated 0 bytes in 1362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:43.619199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:43.621268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:43.639647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889605-7ba3-43d2-ae8f-b0ab66f077f3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:43.732952 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:43.733970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:43.734826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:43.735131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] PATH_INFO: `/v3/projects/ee7ac9ab55514dbea4f8246e43971d2a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:43.736501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7ac9ab55514dbea4f8246e43971d2a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:43.744671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:43.828174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:43.828759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:43.828759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:43.871137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] MFA Rules not processed for user `611f51a2f6434e0ab78559d8b03493df`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:43.872821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:43.896295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7ac9ab55514dbea4f8246e43971d2a', 'request_id': 'req-f9087aa7-514e-4865-a6eb-d85a4aba84c3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:43.939407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7ac9ab55514dbea4f8246e43971d2a', 'request_id': 'req-f9087aa7-514e-4865-a6eb-d85a4aba84c3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:43.981280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1124/2223] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:43.993549 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:43.999468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:44.031547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7ac9ab55514dbea4f8246e43971d2a', 'request_id': 'req-f9087aa7-514e-4865-a6eb-d85a4aba84c3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:44.045856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] The token cache is being invalidate because project ee7ac9ab55514dbea4f8246e43971d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:44.046461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:44.048284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f9087aa7-514e-4865-a6eb-d85a4aba84c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:44.053560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1100/2224] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:43 2022] DELETE /identity/v3/projects/ee7ac9ab55514dbea4f8246e43971d2a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:44.057828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982e8e-7e17-4f9d-80d7-304990052db7 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.058069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982e8e-7e17-4f9d-80d7-304990052db7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.058446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982e8e-7e17-4f9d-80d7-304990052db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.090787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475d0b7-51ab-4573-b4f8-f943a142e774 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:44.093350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-83982e8e-7e17-4f9d-80d7-304990052db7 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:44.095103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-83982e8e-7e17-4f9d-80d7-304990052db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:44.100531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d16e6-0ada-432d-97a7-c26df77cc696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:44.100531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d16e6-0ada-432d-97a7-c26df77cc696 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.100531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d16e6-0ada-432d-97a7-c26df77cc696 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.100531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d16e6-0ada-432d-97a7-c26df77cc696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.100961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-686d16e6-0ada-432d-97a7-c26df77cc696 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:44.110252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d16e6-0ada-432d-97a7-c26df77cc696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:44.111860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d16e6-0ada-432d-97a7-c26df77cc696 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:44.121321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1125/2225] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:49:44.144315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:44.155400 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1101/2226] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:44.163676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d246-ca03-4270-a408-ad5b4971eab0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.163969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d246-ca03-4270-a408-ad5b4971eab0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.164283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d246-ca03-4270-a408-ad5b4971eab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.185084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ede4d246-ca03-4270-a408-ad5b4971eab0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:44.186470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ede4d246-ca03-4270-a408-ad5b4971eab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:44.198901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1126/2227] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:44.200104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83982e8e-7e17-4f9d-80d7-304990052db7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:44.215239 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83982e8e-7e17-4f9d-80d7-304990052db7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:44.221617 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:44.222970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.222970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.222970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.223755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:44.224085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:44.228152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:44.247582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1102/2228] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:49:44.252218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d246-ca03-4270-a408-ad5b4971eab0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:44.263868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d246-ca03-4270-a408-ad5b4971eab0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:44.269923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:44.277571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.277571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.277571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.277571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:44.277571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:44.420556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1127/2229] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:44 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:49:44.422601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:44.435844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edbcce-8be5-48cc-9fcc-8b45f9218ed0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:44.510598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:44.511352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.511713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.512236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.513066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:44.515092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:44.637311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1103/2230] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:44 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:44.641672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:44.648801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:44.652843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020aee0-bf67-4a7f-bf6a-0c1dbb4277cb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:44.675387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:44.675576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.675758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.676035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.676812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:44.678444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:44.700069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1128/2231] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:44.721237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:44.739022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f41da-be1b-4764-a652-6bebebedcd43 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:44.746995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baea386-9436-4f7e-8714-239dd410b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:44.747847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baea386-9436-4f7e-8714-239dd410b222 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.748191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baea386-9436-4f7e-8714-239dd410b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.748539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baea386-9436-4f7e-8714-239dd410b222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.751163 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baea386-9436-4f7e-8714-239dd410b222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:44.753080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baea386-9436-4f7e-8714-239dd410b222 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:44.775614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1104/2232] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:44.780772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:44.790428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06692392-2be2-4610-b3ce-0815a16e7d8d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:44.798230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:44.798230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.798230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.798230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] PATH_INFO: `/v3/projects/2f83a3e9a58141d8b40084486783800d/users/f7e2832e932d40b7ae218c8bf2269ee0/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:44.818931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f83a3e9a58141d8b40084486783800d, user_id=f7e2832e932d40b7ae218c8bf2269ee0, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:44.821860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:44.955999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1129/2233] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:44 2022] PUT /identity/v3/projects/2f83a3e9a58141d8b40084486783800d/users/f7e2832e932d40b7ae218c8bf2269ee0/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:44.966877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b49b-fcc7-435e-9eb7-1f18641def6b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:44.967199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b49b-fcc7-435e-9eb7-1f18641def6b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:44.967450 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b49b-fcc7-435e-9eb7-1f18641def6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:45.008414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ef67b49b-fcc7-435e-9eb7-1f18641def6b None None] MFA Rules not processed for user `f7e2832e932d40b7ae218c8bf2269ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:45.009960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ef67b49b-fcc7-435e-9eb7-1f18641def6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:45.122454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1105/2234] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:45.133873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:45.135670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:45.215749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31460011-9072-45e5-9e92-687b9945fe98 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:45.221519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8b646-17bd-4f16-8567-c3b6a85f07fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:45.221519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b646-17bd-4f16-8567-c3b6a85f07fa service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:45.221823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b646-17bd-4f16-8567-c3b6a85f07fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:45.222230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8b646-17bd-4f16-8567-c3b6a85f07fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:45.232534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8b646-17bd-4f16-8567-c3b6a85f07fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:45.234184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8b646-17bd-4f16-8567-c3b6a85f07fa service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:45.257830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1130/2235] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:45.298821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b49b-fcc7-435e-9eb7-1f18641def6b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:45.369625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b49b-fcc7-435e-9eb7-1f18641def6b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:45.374720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af585e-5058-4162-9573-ca054021054b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:45.375533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af585e-5058-4162-9573-ca054021054b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:45.375936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af585e-5058-4162-9573-ca054021054b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:45.376217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af585e-5058-4162-9573-ca054021054b admin admin] PATH_INFO: `/v3/users/f7e2832e932d40b7ae218c8bf2269ee0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:45.377541 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af585e-5058-4162-9573-ca054021054b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e2832e932d40b7ae218c8bf2269ee0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:45.379913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af585e-5058-4162-9573-ca054021054b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:45.408242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:45.572090 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e2832e932d40b7ae218c8bf2269ee0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:45.591544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e2832e932d40b7ae218c8bf2269ee0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:45.764271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e2832e932d40b7ae218c8bf2269ee0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:45.764271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e2832e932d40b7ae218c8bf2269ee0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:45.764271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e2832e932d40b7ae218c8bf2269ee0'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:45.793584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1106/2236] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:49:45 2022] DELETE /identity/v3/users/f7e2832e932d40b7ae218c8bf2269ee0 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:49:45.799392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fddca8-5bf7-4219-a5a9-a1a371461a56 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:45.799695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fddca8-5bf7-4219-a5a9-a1a371461a56 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:45.800025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fddca8-5bf7-4219-a5a9-a1a371461a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:45.818965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b1fddca8-5bf7-4219-a5a9-a1a371461a56 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:45.826493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b1fddca8-5bf7-4219-a5a9-a1a371461a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:45.898404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1131/2237] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:45.909758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:45.911215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:46.000575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af585e-5058-4162-9573-ca054021054b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:46.014773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:46.016715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:46.016997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:46.017280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:46.026846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:46.028284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:46.038080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1107/2238] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:46.091900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f51886-7eba-4053-a219-ea71e9548d9f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:46.092348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f51886-7eba-4053-a219-ea71e9548d9f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:46.092348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f51886-7eba-4053-a219-ea71e9548d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:46.113541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b2f51886-7eba-4053-a219-ea71e9548d9f None None] MFA Rules not processed for user `611f51a2f6434e0ab78559d8b03493df`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:46.114969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b2f51886-7eba-4053-a219-ea71e9548d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:46.214135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1132/2239] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:46.823714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:46.829344 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:46.865185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c0a51-84b4-4b75-b29b-460c7965c658 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:46.870588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:46.871668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:46.872108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:46.872683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] PATH_INFO: `/v3/projects/2f83a3e9a58141d8b40084486783800d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:46.874068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f83a3e9a58141d8b40084486783800d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:46.876267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:47.043417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f83a3e9a58141d8b40084486783800d', 'request_id': 'req-ea432591-a55a-4d33-9341-6f9f46b54895'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:47.052835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f83a3e9a58141d8b40084486783800d', 'request_id': 'req-ea432591-a55a-4d33-9341-6f9f46b54895'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:47.139134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f83a3e9a58141d8b40084486783800d', 'request_id': 'req-ea432591-a55a-4d33-9341-6f9f46b54895'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:47.145695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:47.152240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] The token cache is being invalidate because project 2f83a3e9a58141d8b40084486783800d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:49:47.153385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:47.155480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ea432591-a55a-4d33-9341-6f9f46b54895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:49:47.161428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1108/2240] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:49:46 2022] DELETE /identity/v3/projects/2f83a3e9a58141d8b40084486783800d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:47.162326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385e98-872e-4f2d-ae21-9725101318a7 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.162769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385e98-872e-4f2d-ae21-9725101318a7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.162891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385e98-872e-4f2d-ae21-9725101318a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.209669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-61385e98-872e-4f2d-ae21-9725101318a7 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:47.210900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-61385e98-872e-4f2d-ae21-9725101318a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:47.247688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:47.316338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1133/2241] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:47.325088 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a859309-992a-4c93-a4c2-ec4fea1a67c8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.325348 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a859309-992a-4c93-a4c2-ec4fea1a67c8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.325636 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a859309-992a-4c93-a4c2-ec4fea1a67c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.349641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5a859309-992a-4c93-a4c2-ec4fea1a67c8 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:47.350849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5a859309-992a-4c93-a4c2-ec4fea1a67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:47.359647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1109/2242] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:47.363695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385e98-872e-4f2d-ae21-9725101318a7 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:47.373967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385e98-872e-4f2d-ae21-9725101318a7 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:47.381321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:47.381989 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.382277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.382558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.383298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:49:47.383620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:47.385278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:47.397891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1134/2243] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:49:47.406625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a859309-992a-4c93-a4c2-ec4fea1a67c8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:47.415899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a859309-992a-4c93-a4c2-ec4fea1a67c8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:47.420664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:47.421325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.421658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.421939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.423640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:47.425656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:47.559528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1110/2244] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:49:47 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:47.560739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:47.572785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40167a-3a2f-43c2-8244-ca37daad2f17 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:47.649266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:47.649984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.650276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.650558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.651269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:47.653334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:47.801095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1135/2245] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:49:47 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:49:47.805896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:47.818589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ceeb2-2fd6-4c80-9b97-2240dfb3b3a9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:47.823272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:47.823948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.824502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.824502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.825283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:47.826929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:47.838734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1111/2246] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:49:47.842613 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:47.853461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63de35-5f00-4b19-a759-763d7f664e18 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:47.858536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64946c-841a-443e-a74a-1d189194eb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:47.859207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64946c-841a-443e-a74a-1d189194eb4c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.859492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64946c-841a-443e-a74a-1d189194eb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.859774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64946c-841a-443e-a74a-1d189194eb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.860532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64946c-841a-443e-a74a-1d189194eb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:47.866557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64946c-841a-443e-a74a-1d189194eb4c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:47.873043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1136/2247] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:49:47.877379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:47.887503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960c27b-bacb-41f7-9706-db2f89b99b66 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:47.892681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:47.893560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:47.893852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:47.894137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] PATH_INFO: `/v3/projects/4c78e372e4d147469bc31a44717a1807/users/0e392abb964c48d1945b50d0f1888137/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:47.908951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c78e372e4d147469bc31a44717a1807, user_id=0e392abb964c48d1945b50d0f1888137, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:47.911674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:47.996466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1112/2248] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:49:47 2022] PUT /identity/v3/projects/4c78e372e4d147469bc31a44717a1807/users/0e392abb964c48d1945b50d0f1888137/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:49:48.068110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c02f7-4b16-4da0-84c8-51d3f54d2479 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:48.068411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c02f7-4b16-4da0-84c8-51d3f54d2479 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:48.068721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c02f7-4b16-4da0-84c8-51d3f54d2479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:48.097493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d87c02f7-4b16-4da0-84c8-51d3f54d2479 None None] MFA Rules not processed for user `0e392abb964c48d1945b50d0f1888137`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:48.099054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d87c02f7-4b16-4da0-84c8-51d3f54d2479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:48.205534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1137/2249] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:48.216297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:48.218061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:48.306277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede6c02-737e-4ad8-8f82-ebb25dd6df40 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:48.310129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf72f1-9220-4b25-822c-f65f093e533d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:48.310784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf72f1-9220-4b25-822c-f65f093e533d service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:48.311076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf72f1-9220-4b25-822c-f65f093e533d service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:48.311362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf72f1-9220-4b25-822c-f65f093e533d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:48.325600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf72f1-9220-4b25-822c-f65f093e533d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:48.328716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf72f1-9220-4b25-822c-f65f093e533d service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:48.340688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1113/2250] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:50.257589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f2303-03bb-4721-b104-e3730df750da None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:50.258435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f2303-03bb-4721-b104-e3730df750da None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:50.258737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f2303-03bb-4721-b104-e3730df750da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:50.277082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dd3f2303-03bb-4721-b104-e3730df750da None None] MFA Rules not processed for user `0e392abb964c48d1945b50d0f1888137`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:50.278302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dd3f2303-03bb-4721-b104-e3730df750da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:50.285949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1138/2251] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:49:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:51.192000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21c3a-ca5d-44b6-af6c-7af6efe038a7 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:51.193350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21c3a-ca5d-44b6-af6c-7af6efe038a7 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:51.194167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21c3a-ca5d-44b6-af6c-7af6efe038a7 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:51.196144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1114/2252] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:49:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:49:51.203594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cae3d1-e896-4113-86ae-c572899560a9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:51.204290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cae3d1-e896-4113-86ae-c572899560a9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:51.204945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cae3d1-e896-4113-86ae-c572899560a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:51.231968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:51.246363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-00cae3d1-e896-4113-86ae-c572899560a9 None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:51.250697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-00cae3d1-e896-4113-86ae-c572899560a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:51.335582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1139/2253] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:49:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:51.359312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a21c3a-ca5d-44b6-af6c-7af6efe038a7 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:51.361848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-92a21c3a-ca5d-44b6-af6c-7af6efe038a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:51.446918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a21c3a-ca5d-44b6-af6c-7af6efe038a7 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:51.450513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d17b37-242a-4fc6-80ed-eda03cf09873 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:51.451161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d17b37-242a-4fc6-80ed-eda03cf09873 service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:51.451451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d17b37-242a-4fc6-80ed-eda03cf09873 service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:51.451735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d17b37-242a-4fc6-80ed-eda03cf09873 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:51.462511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d17b37-242a-4fc6-80ed-eda03cf09873 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:51.463981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d17b37-242a-4fc6-80ed-eda03cf09873 service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:51.476086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1115/2254] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:51.768335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace004d-1373-49f1-8351-65fc4939716e None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:51.769353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace004d-1373-49f1-8351-65fc4939716e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:51.769353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace004d-1373-49f1-8351-65fc4939716e None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:51.770752 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1140/2255] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:49:51.777082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a497a-58db-4483-bcc4-f2aa0b2c3aa3 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:51.777895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a497a-58db-4483-bcc4-f2aa0b2c3aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:51.778878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a497a-58db-4483-bcc4-f2aa0b2c3aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:51.805334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3d2a497a-58db-4483-bcc4-f2aa0b2c3aa3 None None] MFA Rules not processed for user `35d3862fe9e846f18d8bc3a0fa5e993a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:51.806648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3d2a497a-58db-4483-bcc4-f2aa0b2c3aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:51.842382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:49:51.865330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1116/2256] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:49:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:49:51.877822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace004d-1373-49f1-8351-65fc4939716e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:51.880193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1ace004d-1373-49f1-8351-65fc4939716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:51.895872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace004d-1373-49f1-8351-65fc4939716e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:51.900485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed627aef-6210-4cad-812c-445398eb5f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:51.901349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627aef-6210-4cad-812c-445398eb5f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:51.901710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627aef-6210-4cad-812c-445398eb5f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:51.902222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed627aef-6210-4cad-812c-445398eb5f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:51.911394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed627aef-6210-4cad-812c-445398eb5f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:51.913169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed627aef-6210-4cad-812c-445398eb5f4c service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:51.925635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1141/2257] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:52.075440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcece5da-e0f3-4cc6-a843-3a09ecb142c3 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:52.076029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcece5da-e0f3-4cc6-a843-3a09ecb142c3 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:52.076416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcece5da-e0f3-4cc6-a843-3a09ecb142c3 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:52.080585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1117/2258] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:49:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 17:49:52.089924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65594200-d51d-4fdb-b8fd-1fc5c145dfe1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:52.094111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65594200-d51d-4fdb-b8fd-1fc5c145dfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:52.094111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65594200-d51d-4fdb-b8fd-1fc5c145dfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:52.138019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-65594200-d51d-4fdb-b8fd-1fc5c145dfe1 None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:52.150340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-65594200-d51d-4fdb-b8fd-1fc5c145dfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:52.176358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1142/2259] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:49:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:49:52.191537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcece5da-e0f3-4cc6-a843-3a09ecb142c3 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:49:52.195162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bcece5da-e0f3-4cc6-a843-3a09ecb142c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:52.217352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcece5da-e0f3-4cc6-a843-3a09ecb142c3 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:49:52.222373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:49:52.223407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:52.223911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:52.231104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:52.253497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:49:52.255660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:49:52.268399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1118/2260] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:49:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:49:59.917081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:49:59.917538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:49:59.917655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:49:59.935049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:49:59.936247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:49:59.992014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1143/2261] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:49:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:00.004805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:00.006267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:00.076390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c327c-6d25-4443-94f2-838281402322 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:00.082125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c34a3f-2b3c-467c-9f43-a216dd57f062 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:00.082808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c34a3f-2b3c-467c-9f43-a216dd57f062 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:00.083098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c34a3f-2b3c-467c-9f43-a216dd57f062 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:00.083378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c34a3f-2b3c-467c-9f43-a216dd57f062 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:00.091518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c34a3f-2b3c-467c-9f43-a216dd57f062 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:00.093780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c34a3f-2b3c-467c-9f43-a216dd57f062 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:00.102189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1119/2262] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:50:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:50:09.885526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:09.887115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:09.955569 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fbfc5-85dd-4e59-9718-6d75e767def9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:09.960475 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:09.961392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:09.961679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:09.961957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] PATH_INFO: `/v3/users/a5fc663eb6fe425da20252c99e079d5b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:09.963186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fc663eb6fe425da20252c99e079d5b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:09.965177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:10.036385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:10.099288 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=44 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:10.158055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fc663eb6fe425da20252c99e079d5b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:10.169955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fc663eb6fe425da20252c99e079d5b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:10.172013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fc663eb6fe425da20252c99e079d5b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:10.187558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fc663eb6fe425da20252c99e079d5b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:10.190294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fc663eb6fe425da20252c99e079d5b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:10.275680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1144/2263] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:50:09 2022] DELETE /identity/v3/users/a5fc663eb6fe425da20252c99e079d5b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:10.280763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:10.281071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:10.281363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:10.298571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:10.299784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:10.358489 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1120/2264] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:10.368692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:10.370336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:10.443155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261aad31-c369-497d-b015-631a606a7716 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:10.446939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357226dd-2d39-466b-ba82-ede8fc3a36b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:10.447599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357226dd-2d39-466b-ba82-ede8fc3a36b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:10.447939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357226dd-2d39-466b-ba82-ede8fc3a36b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:10.448276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357226dd-2d39-466b-ba82-ede8fc3a36b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:10.459085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357226dd-2d39-466b-ba82-ede8fc3a36b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:10.460560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357226dd-2d39-466b-ba82-ede8fc3a36b8 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:10.471639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1145/2265] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:50:10.946152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:10.947578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:10.959815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0d869-7f17-4540-be17-8afe93579edf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:10.977497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:10.977497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:10.982325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:10.982325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] PATH_INFO: `/v3/projects/ba6c36205c23430aa0de7e8b90e91224` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:10.982325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6c36205c23430aa0de7e8b90e91224)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:10.982325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:11.142168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6c36205c23430aa0de7e8b90e91224', 'request_id': 'req-c76a8ffb-c921-4d51-a881-7803c0da2e0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:11.149873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6c36205c23430aa0de7e8b90e91224', 'request_id': 'req-c76a8ffb-c921-4d51-a881-7803c0da2e0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:11.262866 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6c36205c23430aa0de7e8b90e91224', 'request_id': 'req-c76a8ffb-c921-4d51-a881-7803c0da2e0d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:11.275892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] The token cache is being invalidate because project ba6c36205c23430aa0de7e8b90e91224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:50:11.276305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:11.277079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c76a8ffb-c921-4d51-a881-7803c0da2e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:11.279149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1121/2266] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:50:10 2022] DELETE /identity/v3/projects/ba6c36205c23430aa0de7e8b90e91224 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:11.284617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb695-fabd-431d-a3b6-ab6a8d3b9eb1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.285107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb695-fabd-431d-a3b6-ab6a8d3b9eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.285223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296eb695-fabd-431d-a3b6-ab6a8d3b9eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.303965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-296eb695-fabd-431d-a3b6-ab6a8d3b9eb1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:11.305227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-296eb695-fabd-431d-a3b6-ab6a8d3b9eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:11.365593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1146/2267] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:11.380677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380588-9271-4d0c-9ebc-7d452eb45d7b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.380970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380588-9271-4d0c-9ebc-7d452eb45d7b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.381249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380588-9271-4d0c-9ebc-7d452eb45d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.400837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-96380588-9271-4d0c-9ebc-7d452eb45d7b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:11.402031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-96380588-9271-4d0c-9ebc-7d452eb45d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:11.409844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1122/2268] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:11.414038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eb695-fabd-431d-a3b6-ab6a8d3b9eb1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:11.423800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296eb695-fabd-431d-a3b6-ab6a8d3b9eb1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:11.428386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:11.429075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.429365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.429645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.430372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:50:11.430682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:11.432277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:11.444348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1147/2269] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:50:11.448443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96380588-9271-4d0c-9ebc-7d452eb45d7b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:11.458123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96380588-9271-4d0c-9ebc-7d452eb45d7b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:11.462694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:11.463339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.463622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.463896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.464575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:11.466507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:11.562075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1123/2270] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:50:11 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:50:11.563639 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:11.576348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4694db-e9ef-467d-afc4-b7b7068620aa admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:11.614630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:11.645903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:11.646670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.647050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.647426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.648226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:11.650310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:11.727757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1148/2271] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:50:11 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:50:11.731971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:11.743889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e6c2-e308-43e1-9e45-332daf384154 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:11.749281 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:11.749858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.750247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.750699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.751672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:11.753857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:11.766536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1124/2272] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:50:11.770611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:11.781359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722adc6-cdc7-4b53-a2e9-0263a3875b9e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:11.786764 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4570eb-70e4-4bf6-a763-5ffa45b8b099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:11.787429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4570eb-70e4-4bf6-a763-5ffa45b8b099 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.787716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4570eb-70e4-4bf6-a763-5ffa45b8b099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.787995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4570eb-70e4-4bf6-a763-5ffa45b8b099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.788800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4570eb-70e4-4bf6-a763-5ffa45b8b099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:11.794431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4570eb-70e4-4bf6-a763-5ffa45b8b099 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:11.801699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1149/2273] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:50:11.805954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:11.816337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d46f1b-774d-4cf6-8a32-b192bf30ef95 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:11.820955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:11.821824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.822111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.822391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] PATH_INFO: `/v3/projects/7c267a948d82491bb17ddd4c13eb9ba1/users/073445bc675b40d098d6b21e1be59e6e/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:11.836457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c267a948d82491bb17ddd4c13eb9ba1, user_id=073445bc675b40d098d6b21e1be59e6e, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:11.839171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:11.979080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1125/2274] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:50:11 2022] PUT /identity/v3/projects/7c267a948d82491bb17ddd4c13eb9ba1/users/073445bc675b40d098d6b21e1be59e6e/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:11.991005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:11.991407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:11.991694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:12.018572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] MFA Rules not processed for user `073445bc675b40d098d6b21e1be59e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:12.019797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:12.082847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1150/2275] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:12.094544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:12.096225 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:12.118557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:12.153435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025b714-3cd2-4359-8ebe-88d0b0561642 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:12.157708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:12.158346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:12.158629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:12.158905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:12.167111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:12.168552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:12.177877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1126/2276] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:50:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:50:14.448557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:14.449983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:14.522306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8032ac-e0e1-4821-83c9-1b9dc7e84664 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:14.527425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac196b38-3c03-45d2-808d-0301ab04d7f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:14.528110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196b38-3c03-45d2-808d-0301ab04d7f9 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:14.528398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196b38-3c03-45d2-808d-0301ab04d7f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:14.528750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196b38-3c03-45d2-808d-0301ab04d7f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:14.531388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ac196b38-3c03-45d2-808d-0301ab04d7f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:14.542916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac196b38-3c03-45d2-808d-0301ab04d7f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:14.544514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac196b38-3c03-45d2-808d-0301ab04d7f9 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:14.555081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1151/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:50:21.382766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:21.444610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565e48d-12f4-44bd-b260-cf1c19fde735 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:21.449208 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:21.450084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:21.450386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:21.450662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] PATH_INFO: `/v3/users/073445bc675b40d098d6b21e1be59e6e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:21.452439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073445bc675b40d098d6b21e1be59e6e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:21.454443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:21.673664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073445bc675b40d098d6b21e1be59e6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:21.682282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073445bc675b40d098d6b21e1be59e6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:21.772982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073445bc675b40d098d6b21e1be59e6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:21.774123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073445bc675b40d098d6b21e1be59e6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:21.775061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073445bc675b40d098d6b21e1be59e6e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:21.801661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1127/2278] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:50:21 2022] DELETE /identity/v3/users/073445bc675b40d098d6b21e1be59e6e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:21.806790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:21.806790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:21.806790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:21.830570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:21.831922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:21.872542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:21.907082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1152/2279] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:21.917220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:21.918583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:21.964924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:21.976981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7a81f-3c2e-4dca-8c51-1b2b3f66e553 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:21.980798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e8e88-8535-4248-828c-46b221f5967a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:21.984039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8e88-8535-4248-828c-46b221f5967a service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:21.984039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8e88-8535-4248-828c-46b221f5967a service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:21.984039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8e88-8535-4248-828c-46b221f5967a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:21.992226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e8e88-8535-4248-828c-46b221f5967a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:21.993992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e8e88-8535-4248-828c-46b221f5967a service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:22.005359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1128/2280] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:50:22.413330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:22.415071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:22.430434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a7c54-3f18-4a7f-befc-deb1e225c345 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:22.438470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:22.439383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:22.439697 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:22.440016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] PATH_INFO: `/v3/projects/7c267a948d82491bb17ddd4c13eb9ba1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:22.441200 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c267a948d82491bb17ddd4c13eb9ba1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:22.443197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:22.560577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c267a948d82491bb17ddd4c13eb9ba1', 'request_id': 'req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:22.567508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c267a948d82491bb17ddd4c13eb9ba1', 'request_id': 'req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:22.573964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c267a948d82491bb17ddd4c13eb9ba1', 'request_id': 'req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:22.670342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] The token cache is being invalidate because project 7c267a948d82491bb17ddd4c13eb9ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:50:22.670809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:22.671539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2165f3fe-d48a-4ed6-aed1-c79ba7d05a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:22.673376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1153/2281] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:50:22 2022] DELETE /identity/v3/projects/7c267a948d82491bb17ddd4c13eb9ba1 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:22.678573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59832-c7d7-40c8-a338-b49313570f05 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:22.679035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59832-c7d7-40c8-a338-b49313570f05 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:22.679232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59832-c7d7-40c8-a338-b49313570f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:22.697402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-14b59832-c7d7-40c8-a338-b49313570f05 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:22.698616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-14b59832-c7d7-40c8-a338-b49313570f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:22.760881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1129/2282] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:22.769547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7563f-ad01-4723-94ba-cbe96c8bb92b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:22.769840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7563f-ad01-4723-94ba-cbe96c8bb92b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:22.770121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7563f-ad01-4723-94ba-cbe96c8bb92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:22.790041 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-16f7563f-ad01-4723-94ba-cbe96c8bb92b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:22.791250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-16f7563f-ad01-4723-94ba-cbe96c8bb92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:22.799148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1154/2283] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:22.803393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b59832-c7d7-40c8-a338-b49313570f05 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:22.813564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b59832-c7d7-40c8-a338-b49313570f05 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:22.818112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:22.818767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:22.819054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:22.819333 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:22.820084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:50:22.820393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:22.822040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:22.834396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1130/2284] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:50:22.838770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7563f-ad01-4723-94ba-cbe96c8bb92b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:22.848111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7563f-ad01-4723-94ba-cbe96c8bb92b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:22.852936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:22.853585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:22.853865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:22.854140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:22.854840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:22.856906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:22.940689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1155/2285] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:50:22 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:50:22.944937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:22.955965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e146647-64f0-4d57-ae38-1e85657f09a7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:23.030365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:23.031328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:23.031616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:23.031892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:23.032644 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:23.035092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:23.161936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1131/2286] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:50:22 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:50:23.166280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:23.177403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0c2dd-5b37-4265-99d7-1cc4ed90d422 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:23.181925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:23.182576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:23.188461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:23.188461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:23.188461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:23.188461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:23.197422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1156/2287] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:50:23.201327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:23.212246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd68751-6ce7-4130-9a17-c3db2257df84 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:23.216913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17be4a1-cb1e-4e8b-9d24-5d20f016f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:23.218441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be4a1-cb1e-4e8b-9d24-5d20f016f603 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:23.218738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be4a1-cb1e-4e8b-9d24-5d20f016f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:23.219046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17be4a1-cb1e-4e8b-9d24-5d20f016f603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:23.219789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17be4a1-cb1e-4e8b-9d24-5d20f016f603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:23.221486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17be4a1-cb1e-4e8b-9d24-5d20f016f603 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:23.232271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1132/2288] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:50:23.236304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:23.246583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96a5d4-5c84-4e2a-8c5a-c5ea45f2fe54 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:23.251779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:23.252688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:23.252973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:23.253540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] PATH_INFO: `/v3/projects/8d4d85b85edb4aa3b18b88ae45c3b702/users/e87a32eb7afd4eb5ae5c6cdbf06c65d1/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:23.268825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4d85b85edb4aa3b18b88ae45c3b702, user_id=e87a32eb7afd4eb5ae5c6cdbf06c65d1, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:23.271775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:23.333281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1157/2289] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:50:23 2022] PUT /identity/v3/projects/8d4d85b85edb4aa3b18b88ae45c3b702/users/e87a32eb7afd4eb5ae5c6cdbf06c65d1/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:23.343481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899e3a-9db9-46a6-b7bc-8aad4e61f880 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:23.343777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899e3a-9db9-46a6-b7bc-8aad4e61f880 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:23.344060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899e3a-9db9-46a6-b7bc-8aad4e61f880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:23.369601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-00899e3a-9db9-46a6-b7bc-8aad4e61f880 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:23.370972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-00899e3a-9db9-46a6-b7bc-8aad4e61f880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:23.422148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:23.437212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1133/2290] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:50:23.447754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:23.449818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:23.512019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82be4ba-0a20-42ab-9380-3fe4ddca293a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:23.515961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:23.516646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:23.516934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:23.517211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:23.523619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:23.527444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:23.528907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:23.539567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1158/2291] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:50:29.855623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c20e-9605-4092-8acf-cd7be45ebeaa None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:29.856160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c20e-9605-4092-8acf-cd7be45ebeaa None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:29.856339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711c20e-9605-4092-8acf-cd7be45ebeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:29.885574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e711c20e-9605-4092-8acf-cd7be45ebeaa None None] MFA Rules not processed for user `e87a32eb7afd4eb5ae5c6cdbf06c65d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:29.886790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e711c20e-9605-4092-8acf-cd7be45ebeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:29.963284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1134/2292] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:29.967580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:29.992176 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c37f18-a287-44d2-9cc8-ed7990c90e6d service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:29.999937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:30.001538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.002107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.002675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.003925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:30.008032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:30.117138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1159/2293] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:50:29 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:50:30.118459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711c20e-9605-4092-8acf-cd7be45ebeaa None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:30.130271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711c20e-9605-4092-8acf-cd7be45ebeaa None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:30.192168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:30.192890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.193182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.193471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.194181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:30.196170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:30.304647 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1135/2294] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:50:30 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:50:30.308684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:30.320234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e902b8-d8c3-458e-b5ac-c1805481f683 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:30.325029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b7f03-e801-4c8b-845f-6bdd22355889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:30.325688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7f03-e801-4c8b-845f-6bdd22355889 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.325977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7f03-e801-4c8b-845f-6bdd22355889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.326827 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7f03-e801-4c8b-845f-6bdd22355889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.327619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b7f03-e801-4c8b-845f-6bdd22355889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:30.329268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b7f03-e801-4c8b-845f-6bdd22355889 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:30.341862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1160/2295] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:50:30.345869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:30.356247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5716c98-e415-4ddc-a8a0-d5b01db4c8fc admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:30.361734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:30.363401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.363401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.363401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] PATH_INFO: `/v3/projects/f1f01c4526b74cd19bb67a4e1ea98c5c/users/29ecf13bb4d647ed9ce6396f8a072ad3/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.383230 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f01c4526b74cd19bb67a4e1ea98c5c, user_id=29ecf13bb4d647ed9ce6396f8a072ad3, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:30.384071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:30.389454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.389839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.390187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.408811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:30.410055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:30.473962 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1136/2296] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:50:30 2022] PUT /identity/v3/projects/f1f01c4526b74cd19bb67a4e1ea98c5c/users/29ecf13bb4d647ed9ce6396f8a072ad3/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:30.477893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:30.485180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1161/2297] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:50:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:50:30.519965 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:30.521618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:30.540158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:30.556826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254ccb4-db4b-435d-a41e-d6cd29cd4330 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:30.563099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:30.567001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.567001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.567001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.567001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:30.567001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:30.581050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1137/2298] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:50:30.582444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:30.593915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142bf92-f376-4598-932d-3e4d77dfa8e6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:30.607336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008687b8-8102-4e8d-a535-14ee643d69db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:30.607336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008687b8-8102-4e8d-a535-14ee643d69db admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.607336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008687b8-8102-4e8d-a535-14ee643d69db admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.607336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008687b8-8102-4e8d-a535-14ee643d69db admin admin] PATH_INFO: `/v3/projects/f1f01c4526b74cd19bb67a4e1ea98c5c/users/29ecf13bb4d647ed9ce6396f8a072ad3/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.607673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008687b8-8102-4e8d-a535-14ee643d69db admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f01c4526b74cd19bb67a4e1ea98c5c, user_id=29ecf13bb4d647ed9ce6396f8a072ad3, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:30.607748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ec8e1-9f69-4d29-a5b5-8fd019fc613a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:30.607840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008687b8-8102-4e8d-a535-14ee643d69db admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:30.619210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1138/2299] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:50:30 2022] PUT /identity/v3/projects/f1f01c4526b74cd19bb67a4e1ea98c5c/users/29ecf13bb4d647ed9ce6396f8a072ad3/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:30.673601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:30.674299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:30.674586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:30.674870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:30.684061 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:30.758065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:30.760023 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:30.770666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1162/2300] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:50:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 17:50:31.741591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a7c85-3976-4b5f-8578-cea67366031f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:31.742034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a7c85-3976-4b5f-8578-cea67366031f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:31.742354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a7c85-3976-4b5f-8578-cea67366031f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:31.761565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-310a7c85-3976-4b5f-8578-cea67366031f None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:31.763004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-310a7c85-3976-4b5f-8578-cea67366031f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:31.822408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1139/2301] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 15 17:50:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:31.835410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:31.846632 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e79484-282a-4949-9e97-ab000f746af0 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:31.911718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:31.912370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:31.912693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:31.912964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:31.922248 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:31.923830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:31.940637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1163/2302] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:50:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:50:32.297458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6e6e-99ea-45ea-9bba-888cdd1b05cb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:32.298214 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6e6e-99ea-45ea-9bba-888cdd1b05cb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:32.298676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6e6e-99ea-45ea-9bba-888cdd1b05cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:32.355003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-abfd6e6e-99ea-45ea-9bba-888cdd1b05cb None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:32.358966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-abfd6e6e-99ea-45ea-9bba-888cdd1b05cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:32.388424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1140/2303] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 15 17:50:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:50:32.392760 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:32.404131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc3365-ad09-40b9-a745-50f8d4e1b3b7 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:32.425587 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:32.427165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:32.427165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:32.427165 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:32.441043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:32.443214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:32.456730 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1164/2304] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:50:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:50:32.708490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af6f30-bb6f-4e64-87a7-d651d41e1c00 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:32.708814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af6f30-bb6f-4e64-87a7-d651d41e1c00 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:32.709098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af6f30-bb6f-4e64-87a7-d651d41e1c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:32.736166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c6af6f30-bb6f-4e64-87a7-d651d41e1c00 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:32.737623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c6af6f30-bb6f-4e64-87a7-d651d41e1c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:32.751811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1141/2305] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:50:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:32.766225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:32.783037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc6fda-ead1-43fe-a765-2a8c1882be6b service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:32.783644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:32.784545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:32.785095 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:32.785571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:32.795407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:32.797365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:32.809620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1165/2306] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:50:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:50:33.809956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a402d9-01a1-4920-9352-0c13255d179c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:33.809956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a402d9-01a1-4920-9352-0c13255d179c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:33.809956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a402d9-01a1-4920-9352-0c13255d179c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:33.825000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-15a402d9-01a1-4920-9352-0c13255d179c None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:33.826627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-15a402d9-01a1-4920-9352-0c13255d179c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:33.835343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1142/2307] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 15 17:50:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:33.848762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:33.860919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a64d-c1d3-4dbc-8af8-3db7dc3adb31 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:33.868146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:33.868961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:33.869326 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:33.869694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:33.879916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:33.881962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:33.891487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1166/2308] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:50:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:50:37.454401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:37.454955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:37.454955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:37.487346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] MFA Rules not processed for user `29ecf13bb4d647ed9ce6396f8a072ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:37.488589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:37.510624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:37.510624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:37.580859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1143/2309] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:37.595918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:37.596221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d49d8-12e8-459e-8c98-e000d7410161 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:37.597625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:37.601687 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:37.603039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:37.603496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:37.604516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] PATH_INFO: `/v3/users/0e392abb964c48d1945b50d0f1888137` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:37.606411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e392abb964c48d1945b50d0f1888137)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:37.609148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:37.611827 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:37.657424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:37.698323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbabb07-ea40-4a6c-bba7-d67f4cd4cd33 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:37.704437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6120345-afbb-432a-b251-2a93aa5c21aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:37.705142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6120345-afbb-432a-b251-2a93aa5c21aa service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:37.705426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6120345-afbb-432a-b251-2a93aa5c21aa service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:37.705699 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6120345-afbb-432a-b251-2a93aa5c21aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:37.770311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6120345-afbb-432a-b251-2a93aa5c21aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:37.771947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6120345-afbb-432a-b251-2a93aa5c21aa service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:37.783486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1144/2310] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:50:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:50:37.896236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e392abb964c48d1945b50d0f1888137'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:37.909124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e392abb964c48d1945b50d0f1888137'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:37.909825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e392abb964c48d1945b50d0f1888137'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:37.918359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e392abb964c48d1945b50d0f1888137'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:37.921047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e392abb964c48d1945b50d0f1888137'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:38.018941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1167/2311] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:50:37 2022] DELETE /identity/v3/users/0e392abb964c48d1945b50d0f1888137 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:38.023555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:38.023857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:38.024142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:38.042578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:38.044447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:38.107605 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1145/2312] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:38.117294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:38.118747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:38.199454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8b3b-e6ba-4280-98a1-177344ecec5e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:38.204299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2080484-452f-4d2e-9c07-0a58a5ab2c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:38.205348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2080484-452f-4d2e-9c07-0a58a5ab2c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:38.205810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2080484-452f-4d2e-9c07-0a58a5ab2c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:38.206231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2080484-452f-4d2e-9c07-0a58a5ab2c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:38.218841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2080484-452f-4d2e-9c07-0a58a5ab2c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:38.221075 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2080484-452f-4d2e-9c07-0a58a5ab2c14 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:38.236480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1168/2313] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:50:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:50:38.497526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:50:38.497526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:38.504047 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42727b12-68ee-489e-9aca-862d779b6fe2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:50:38.510580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:50:38.512220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:38.512354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:38.512874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] PATH_INFO: `/v3/projects/4c78e372e4d147469bc31a44717a1807` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:38.515471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c78e372e4d147469bc31a44717a1807)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:50:38.518670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:50:38.660164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c78e372e4d147469bc31a44717a1807', 'request_id': 'req-8410d9a6-59fc-4bb5-a894-bf98e35fc310'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:38.666989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c78e372e4d147469bc31a44717a1807', 'request_id': 'req-8410d9a6-59fc-4bb5-a894-bf98e35fc310'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:38.754436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c78e372e4d147469bc31a44717a1807', 'request_id': 'req-8410d9a6-59fc-4bb5-a894-bf98e35fc310'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:38.764684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:50:38.769104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] The token cache is being invalidate because project 4c78e372e4d147469bc31a44717a1807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:50:38.769609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:38.771197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8410d9a6-59fc-4bb5-a894-bf98e35fc310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:50:38.773537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1146/2314] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:50:38 2022] DELETE /identity/v3/projects/4c78e372e4d147469bc31a44717a1807 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:50:40.487534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8d8a-2dc5-4902-8beb-d4bacc87e23d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:50:40.488112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8d8a-2dc5-4902-8beb-d4bacc87e23d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:50:40.488112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8d8a-2dc5-4902-8beb-d4bacc87e23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:50:40.507842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-b59d8d8a-2dc5-4902-8beb-d4bacc87e23d None None] MFA Rules not processed for user `e87a32eb7afd4eb5ae5c6cdbf06c65d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:50:40.509215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b59d8d8a-2dc5-4902-8beb-d4bacc87e23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:50:40.570857 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1169/2315] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:50:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:50:50.563565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=77,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:51:16.818870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5c0-f7cb-4e07-9733-2407f428f803 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:51:16.819455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5c0-f7cb-4e07-9733-2407f428f803 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:51:16.819455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5c0-f7cb-4e07-9733-2407f428f803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:51:16.837139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b932c5c0-f7cb-4e07-9733-2407f428f803 None None] MFA Rules not processed for user `34e7019b46d743a1b6b0d3315e69053f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:51:16.838342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b932c5c0-f7cb-4e07-9733-2407f428f803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:51:16.895160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1147/2316] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:51:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:51:21.125586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:51:21.126092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:51:21.126190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:51:21.144508 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] MFA Rules not processed for user `8873b1c9f58a4adea93ebe21f5f3263d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:51:21.145743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:51:21.203253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1170/2317] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:51:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:51:24.189841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b8d6-9b97-4037-a561-90ccd62d0a24 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:51:24.190395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b8d6-9b97-4037-a561-90ccd62d0a24 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:51:24.190494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69b8d6-9b97-4037-a561-90ccd62d0a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:51:24.209470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2d69b8d6-9b97-4037-a561-90ccd62d0a24 None None] MFA Rules not processed for user `611f51a2f6434e0ab78559d8b03493df`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:51:24.210788 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2d69b8d6-9b97-4037-a561-90ccd62d0a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:51:24.280194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1148/2318] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:51:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:51:30.358042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:51:30.362388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:51:30.424744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5797eb-d95b-4fef-ac23-03aa5582f8d5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:51:30.428534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:51:30.429243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:51:30.429525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:51:30.444239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:51:30.444239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:51:30.523709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:51:30.525342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:51:30.538204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1171/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:51:34.268049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=50 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:51:40.533215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=75 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:51:42.321736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:51:42.322193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:51:42.322298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:51:42.340523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] MFA Rules not processed for user `caab1412244c43adac226cb253a10880`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:51:42.341774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:51:42.396873 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1149/2320] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:51:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:51:52.390103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=25 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:06.906873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:06.910897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:06.966304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351dfc-7ad6-48ed-9fbd-29b164aebc2c service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:06.969977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:06.970627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:06.970913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:06.971188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:06.973019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:07.055079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:07.056537 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:07.068656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1172/2321] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:11.224673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:11.226099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:11.238176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b69f6c-f067-4605-b886-f78e6a454ee0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:11.241766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:11.242414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:11.242698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:11.242978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:11.244753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:11.259062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:11.260495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:11.271951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1150/2322] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:17.063224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=54 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:21.267086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:22.693378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:22.702444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bc39f-2fdc-4e78-ae7e-e2b3787b9942 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:22.706379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:22.707085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:22.707366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:22.707642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:22.709424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:22.771578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:22.773071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:22.782262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1173/2323] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:27.076900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:27.078329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:27.092505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2edc2-c791-4320-b10e-4bdfbef36cda service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:27.098048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab70811-f1cd-4200-97b6-d13fce5d741b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:27.098686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70811-f1cd-4200-97b6-d13fce5d741b service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:27.098982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70811-f1cd-4200-97b6-d13fce5d741b service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:27.099266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70811-f1cd-4200-97b6-d13fce5d741b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:27.101037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5ab70811-f1cd-4200-97b6-d13fce5d741b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:27.111904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab70811-f1cd-4200-97b6-d13fce5d741b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:27.113544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab70811-f1cd-4200-97b6-d13fce5d741b service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:27.122439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1151/2324] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:32.777978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=28 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:34.560323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:34.572380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:34.579450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ae54-b237-4e13-8a3c-7a5506687302 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:34.585999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:34.586667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:34.586955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:34.587232 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:34.589732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:34.601373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:34.603067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:34.614160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1174/2325] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:37.118365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:37.434292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:37.434586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:37.434871 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:37.452684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] MFA Rules not processed for user `5013f4ba5cef44f7bdab6531ec0b0836`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:52:37.453888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:37.460716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1152/2326] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:52:37.769371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:37.779614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a36f5-1e90-426f-9e75-bc382f9bf6f7 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:37.783286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:37.783923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:37.784204 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:37.784480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:37.786320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:37.796497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:37.798005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:37.805964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1175/2327] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:38.266511 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:38.267953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:38.349186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc2b93-ac83-4463-9e5a-8440086e8cb4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:38.354304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:38.355570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:38.355570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:38.355570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:38.357360 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:38.368664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:38.370844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:38.381040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1153/2328] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:41.882290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:41.891264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dbf5d-8836-4507-9ea6-d4ef7941f98c service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:41.895162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:41.896976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:41.897464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:41.897826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:41.899886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:41.909993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:41.911610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:41.927620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1176/2329] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:42.392984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:42.398869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:42.408826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28c168-9283-4be4-b668-3a351542b3a0 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:42.414683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:42.415801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:42.416432 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:42.416913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:42.419682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:42.433720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:42.435957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:42.447509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1154/2330] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:42.521333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:42.523146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:42.626004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104d856-9dad-4905-877d-b92425a9df9b service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:42.631121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:42.632077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:42.632370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:42.632816 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] PATH_INFO: `/v3/users/611f51a2f6434e0ab78559d8b03493df` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:42.634093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=611f51a2f6434e0ab78559d8b03493df)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:42.637779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:42.894749 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:42.910914 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bfb1a-b9c4-4e58-9a3b-d71462e3099d service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:42.917700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '611f51a2f6434e0ab78559d8b03493df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:42.919476 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:42.922267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:42.922267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:42.922267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:42.923379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:42.941184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '611f51a2f6434e0ab78559d8b03493df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:42.942262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '611f51a2f6434e0ab78559d8b03493df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:42.942850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:42.945730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:42.953383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '611f51a2f6434e0ab78559d8b03493df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:42.954469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '611f51a2f6434e0ab78559d8b03493df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:42.962515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1155/2331] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:43.045434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1177/2332] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:52:42 2022] DELETE /identity/v3/users/611f51a2f6434e0ab78559d8b03493df => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:52:43.302578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:43.314775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca057f01-48a8-4e55-a769-4eed9146b299 service nova] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:43.376380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:43.377294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:43.377584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:43.377869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] PATH_INFO: `/v3/projects/d6aa269b9af048c1860128aa8db2bf28` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:43.379236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6aa269b9af048c1860128aa8db2bf28)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:43.381304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:43.530990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6aa269b9af048c1860128aa8db2bf28', 'request_id': 'req-1d04ec21-2407-46bf-95b3-28620e6f779e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:43.537494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6aa269b9af048c1860128aa8db2bf28', 'request_id': 'req-1d04ec21-2407-46bf-95b3-28620e6f779e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:43.541393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:43.609291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6aa269b9af048c1860128aa8db2bf28', 'request_id': 'req-1d04ec21-2407-46bf-95b3-28620e6f779e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:43.620616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] The token cache is being invalidate because project d6aa269b9af048c1860128aa8db2bf28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:52:43.620916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:43.621814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:43.623719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1156/2333] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:52:43 2022] DELETE /identity/v3/projects/d6aa269b9af048c1860128aa8db2bf28 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:52:47.427718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:47.427718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:47.427718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:47.427718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5838c48-e402-42f8-bb4e-5e56c8a2d529 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:47.465854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:47.466444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:47.466732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:47.467012 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:47.468823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:47.527718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:47.529402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:47.538800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1178/2334] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:49.143391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:49.144830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:49.207061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04ec21-2407-46bf-95b3-28620e6f779e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:49.210932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:49.211579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:49.211864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:49.212143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:49.213996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:49.278003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:49.279454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:49.289146 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1157/2335] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:53.944430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:53.945931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:53.975676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc554b-47be-44e4-bf6c-90fa1912d28e service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:53.981384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:53.982290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:53.982542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:53.982812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] PATH_INFO: `/v3/users/8873b1c9f58a4adea93ebe21f5f3263d` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:53.984018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8873b1c9f58a4adea93ebe21f5f3263d)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:53.986039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:54.178558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8873b1c9f58a4adea93ebe21f5f3263d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.193806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8873b1c9f58a4adea93ebe21f5f3263d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.194749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8873b1c9f58a4adea93ebe21f5f3263d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.203269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:54.205274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8873b1c9f58a4adea93ebe21f5f3263d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.206004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-74685acd-ba57-41ad-852d-a373e3afb991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8873b1c9f58a4adea93ebe21f5f3263d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.274828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1179/2336] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:52:53 2022] DELETE /identity/v3/users/8873b1c9f58a4adea93ebe21f5f3263d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:52:54.614076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:54.629685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b790e-99ec-4557-ba2d-418f88f4ce06 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:54.689832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:54.690691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:54.690973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:54.691247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] PATH_INFO: `/v3/projects/882c7039b37f45f8843752b457525d9b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:54.692470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] RBAC: Authorizing `identity:delete_project(project_id=882c7039b37f45f8843752b457525d9b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:54.694438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:54.813619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:54.821650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882c7039b37f45f8843752b457525d9b', 'request_id': 'req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.828771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882c7039b37f45f8843752b457525d9b', 'request_id': 'req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.912697 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882c7039b37f45f8843752b457525d9b', 'request_id': 'req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.923665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] The token cache is being invalidate because project 882c7039b37f45f8843752b457525d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:52:54.924045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.924789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5818e5f9-aa9e-4e3b-b725-e2e6078d817a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:52:54.926440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1158/2337] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:52:54 2022] DELETE /identity/v3/projects/882c7039b37f45f8843752b457525d9b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:52:54.935338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732f3a-98a0-4694-9510-e6de21b90201 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:54.935338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732f3a-98a0-4694-9510-e6de21b90201 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:54.935338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732f3a-98a0-4694-9510-e6de21b90201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:54.951750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-49732f3a-98a0-4694-9510-e6de21b90201 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:52:54.958375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-49732f3a-98a0-4694-9510-e6de21b90201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:55.015903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1180/2338] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:52:55.023686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434c59-7452-4ea5-87bc-faa12368b133 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.024063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434c59-7452-4ea5-87bc-faa12368b133 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.024419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434c59-7452-4ea5-87bc-faa12368b133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.043249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-53434c59-7452-4ea5-87bc-faa12368b133 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:52:55.044460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-53434c59-7452-4ea5-87bc-faa12368b133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:55.052522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1159/2339] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:52:55.056971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49732f3a-98a0-4694-9510-e6de21b90201 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.067544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49732f3a-98a0-4694-9510-e6de21b90201 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:55.072185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:55.073501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.075460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.075460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.075460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:52:55.075460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:55.076709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:55.088932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1181/2340] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:52:55.093419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53434c59-7452-4ea5-87bc-faa12368b133 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.103231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53434c59-7452-4ea5-87bc-faa12368b133 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:55.110328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:55.110999 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.111287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.111564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.112274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:55.114240 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:55.255297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1160/2341] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:52:55 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:52:55.259589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.270511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a8306-ad77-4c12-a654-d7dfcfc96432 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:55.331773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:55.332442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.332775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.333053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.333769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:55.335769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:55.446950 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1182/2342] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:52:55 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:52:55.452253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.462734 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9626b55-0122-4e90-b795-3b600f0f3f14 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:55.468380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:55.469083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.469368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.469646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.470494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:55.472108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:55.487247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1161/2343] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:52:55.488465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.499642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bac79-9fac-437a-ac28-c69dd6ef1688 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:55.504714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713efe0c-5a62-48db-9e4f-e9b0800706e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:55.506943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713efe0c-5a62-48db-9e4f-e9b0800706e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.506943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713efe0c-5a62-48db-9e4f-e9b0800706e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.506943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713efe0c-5a62-48db-9e4f-e9b0800706e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.506943 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713efe0c-5a62-48db-9e4f-e9b0800706e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:55.508348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713efe0c-5a62-48db-9e4f-e9b0800706e5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:55.518860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1183/2344] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:52:55.523046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.533752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea412602-92bb-4cae-9256-1e4e2b92036d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:55.538565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:55.539455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.539747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.540033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] PATH_INFO: `/v3/projects/aad73694b12e4ed4b2cf5972e7a947ba/users/a1d0202794d64b5f9499e607f7802785/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.555717 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad73694b12e4ed4b2cf5972e7a947ba, user_id=a1d0202794d64b5f9499e607f7802785, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:55.558504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:55.644480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1162/2345] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:52:55 2022] PUT /identity/v3/projects/aad73694b12e4ed4b2cf5972e7a947ba/users/a1d0202794d64b5f9499e607f7802785/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:52:55.655851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c1125-13f2-4631-8338-9e0494a6ac51 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.656533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c1125-13f2-4631-8338-9e0494a6ac51 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.656664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c1125-13f2-4631-8338-9e0494a6ac51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.685224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8f2c1125-13f2-4631-8338-9e0494a6ac51 None None] MFA Rules not processed for user `a1d0202794d64b5f9499e607f7802785`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:52:55.686513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8f2c1125-13f2-4631-8338-9e0494a6ac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:55.731591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:55.756213 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1184/2346] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:52:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:52:55.760893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.824000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc871c-edb8-4ece-b0b4-a4dfdd2f32ed admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:55.828923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:55.829582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:55.829894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:55.830175 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:55.830869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:55.832860 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:55.936567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1163/2347] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:52:55 2022] POST /identity/v3/projects => generated 367 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:52:55.940485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c1125-13f2-4631-8338-9e0494a6ac51 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:55.953934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c1125-13f2-4631-8338-9e0494a6ac51 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:56.028397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:56.029739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.030191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.030629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.035879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:56.035879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:56.119323 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1185/2348] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:52:55 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:52:56.131089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:56.144144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5e963-5d70-4df8-9959-6ad93053b53b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:56.150224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:56.151454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.151755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.152035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.152816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:56.154464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:56.176638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1164/2349] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:52:56.180786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:56.191324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f436-0679-4844-b5dc-2a7ac430698d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:56.196004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:56.196933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.197216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.197493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] PATH_INFO: `/v3/projects/9baa4fa09a24429db0cd7ed8b0684778/users/01a18e201a1c4e74b055b0336ae018cf/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.216107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9baa4fa09a24429db0cd7ed8b0684778, user_id=01a18e201a1c4e74b055b0336ae018cf, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:56.218938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:56.333688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1186/2350] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:52:56 2022] PUT /identity/v3/projects/9baa4fa09a24429db0cd7ed8b0684778/users/01a18e201a1c4e74b055b0336ae018cf/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:52:56.337726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:56.348095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e039a0-6410-40c1-829d-80ab34f31728 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:56.390493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:56.418120 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f7799-27b4-4766-a2c4-1f3de42e9dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:56.418641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f7799-27b4-4766-a2c4-1f3de42e9dad admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.418945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f7799-27b4-4766-a2c4-1f3de42e9dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.419227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f7799-27b4-4766-a2c4-1f3de42e9dad admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.420012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f7799-27b4-4766-a2c4-1f3de42e9dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:56.421820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f7799-27b4-4766-a2c4-1f3de42e9dad admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:56.440796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1165/2351] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:52:56.441657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:56.453515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb745c-3122-4c7d-9bc7-183328a91cdb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:56.459280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:56.460097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.460097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.460097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] PATH_INFO: `/v3/projects/9baa4fa09a24429db0cd7ed8b0684778/users/01a18e201a1c4e74b055b0336ae018cf/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.462931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9baa4fa09a24429db0cd7ed8b0684778, user_id=01a18e201a1c4e74b055b0336ae018cf, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:56.464847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:56.477970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1187/2352] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:52:56 2022] PUT /identity/v3/projects/9baa4fa09a24429db0cd7ed8b0684778/users/01a18e201a1c4e74b055b0336ae018cf/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:52:56.488339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.488811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.489292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.533109 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] MFA Rules not processed for user `01a18e201a1c4e74b055b0336ae018cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:52:56.535137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:56.637937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:56.640209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:56.646231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1166/2353] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:52:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:52:56.661032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:52:56.668750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:56.749173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d8b1a-b03e-47b5-a07c-9c02c49d9f09 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:56.756034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d34715-9d93-4eb6-a656-09f17d027641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:56.756887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34715-9d93-4eb6-a656-09f17d027641 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.757568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34715-9d93-4eb6-a656-09f17d027641 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.757568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34715-9d93-4eb6-a656-09f17d027641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.761424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b1d34715-9d93-4eb6-a656-09f17d027641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:56.761606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3f8af-ebc9-4d0d-a990-24fcf97a3b25 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:52:56.776559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d34715-9d93-4eb6-a656-09f17d027641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:56.778448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d34715-9d93-4eb6-a656-09f17d027641 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:56.789239 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:52:56.789896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:56.790313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:56.790600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:56.794895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1188/2354] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:52:56.820530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:52:56.825395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:52:56.849157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1167/2355] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:52:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:52:59.455860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e113d4-dd58-4bb5-ad51-e481af72e724 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:52:59.456394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e113d4-dd58-4bb5-ad51-e481af72e724 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:52:59.456394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e113d4-dd58-4bb5-ad51-e481af72e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:52:59.479026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d3e113d4-dd58-4bb5-ad51-e481af72e724 None None] MFA Rules not processed for user `34e7019b46d743a1b6b0d3315e69053f`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:52:59.480245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d3e113d4-dd58-4bb5-ad51-e481af72e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:52:59.493896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:52:59.544799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1189/2356] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:00.015102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:00.080576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e3b86-b14f-499f-b21d-d23c05fe0bee service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:00.085134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:00.086005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:00.086330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:00.086608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] PATH_INFO: `/v3/users/a1d0202794d64b5f9499e607f7802785` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:00.087805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d0202794d64b5f9499e607f7802785)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:00.090578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:00.093187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:00.288028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d0202794d64b5f9499e607f7802785'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:00.296870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d0202794d64b5f9499e607f7802785'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:00.375262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d0202794d64b5f9499e607f7802785'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:00.376567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d0202794d64b5f9499e607f7802785'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:00.377387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d0202794d64b5f9499e607f7802785'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:00.407954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1168/2357] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:00 2022] DELETE /identity/v3/users/a1d0202794d64b5f9499e607f7802785 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:00.412502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:00.413029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:00.413423 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:00.431988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:00.433488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:00.495580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1190/2358] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:00.505665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:00.508226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:00.572487 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2bc27-fedb-4eea-a331-ff3bb7bc2203 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:00.576254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:00.577037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:00.577320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:00.577594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:00.587194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:00.589730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:00.598075 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1169/2359] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:01.010593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:01.012019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:01.024478 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466c289-41dc-4e48-9d81-3dabd52d74f0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:01.029726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:01.030643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:01.030937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:01.031209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] PATH_INFO: `/v3/projects/aad73694b12e4ed4b2cf5972e7a947ba` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:01.032406 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad73694b12e4ed4b2cf5972e7a947ba)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:01.034364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:01.165391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad73694b12e4ed4b2cf5972e7a947ba', 'request_id': 'req-cc729c48-ed92-4bef-8694-de5d03fa511d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.171581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad73694b12e4ed4b2cf5972e7a947ba', 'request_id': 'req-cc729c48-ed92-4bef-8694-de5d03fa511d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.177449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad73694b12e4ed4b2cf5972e7a947ba', 'request_id': 'req-cc729c48-ed92-4bef-8694-de5d03fa511d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.261269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] The token cache is being invalidate because project aad73694b12e4ed4b2cf5972e7a947ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:01.264838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.264838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.264838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1191/2360] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:01 2022] DELETE /identity/v3/projects/aad73694b12e4ed4b2cf5972e7a947ba => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:01.267810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:01.278150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0a7e9-6237-4f22-86aa-60ea70ed9349 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:01.334929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:01.335911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:01.336203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:01.336490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] PATH_INFO: `/v3/users/01a18e201a1c4e74b055b0336ae018cf` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:01.337741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a18e201a1c4e74b055b0336ae018cf)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:01.340418 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:01.444961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:01.540145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a18e201a1c4e74b055b0336ae018cf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.548835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a18e201a1c4e74b055b0336ae018cf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.638190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a18e201a1c4e74b055b0336ae018cf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.638810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a18e201a1c4e74b055b0336ae018cf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.639446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8055407e-6911-4052-a725-c89ea03437f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a18e201a1c4e74b055b0336ae018cf'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:01.658287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1170/2361] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:01 2022] DELETE /identity/v3/users/01a18e201a1c4e74b055b0336ae018cf => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:02.113595 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:02.115091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:02.128858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729c48-ed92-4bef-8694-de5d03fa511d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:02.177640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:02.190024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:02.190926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:02.191212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:02.191487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] PATH_INFO: `/v3/projects/9baa4fa09a24429db0cd7ed8b0684778` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:02.192831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9baa4fa09a24429db0cd7ed8b0684778)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:02.196194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:02.354140 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9baa4fa09a24429db0cd7ed8b0684778', 'request_id': 'req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:02.360265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9baa4fa09a24429db0cd7ed8b0684778', 'request_id': 'req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:02.366425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9baa4fa09a24429db0cd7ed8b0684778', 'request_id': 'req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:02.461620 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] The token cache is being invalidate because project 9baa4fa09a24429db0cd7ed8b0684778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:02.461914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:02.473231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0176035d-3b0e-4dcb-b49e-df1d8dc03c9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:02.475243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1192/2362] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:02 2022] DELETE /identity/v3/projects/9baa4fa09a24429db0cd7ed8b0684778 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:02.480779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e8dc-5839-4b4d-beb1-f1e78198bcf2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:02.481313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e8dc-5839-4b4d-beb1-f1e78198bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:02.481602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e8dc-5839-4b4d-beb1-f1e78198bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:02.500995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-5cc5e8dc-5839-4b4d-beb1-f1e78198bcf2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:02.502163 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5cc5e8dc-5839-4b4d-beb1-f1e78198bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:02.578794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1171/2363] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:02.587942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eaad1-227b-42fd-84e1-947b6c96f42c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:02.588367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eaad1-227b-42fd-84e1-947b6c96f42c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:02.588683 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006eaad1-227b-42fd-84e1-947b6c96f42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:02.608552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-006eaad1-227b-42fd-84e1-947b6c96f42c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:02.610050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-006eaad1-227b-42fd-84e1-947b6c96f42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:02.618765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1193/2364] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:02.623198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5e8dc-5839-4b4d-beb1-f1e78198bcf2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:02.633219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5e8dc-5839-4b4d-beb1-f1e78198bcf2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:02.639321 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:02.640012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:02.640301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:02.640629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:02.641439 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:53:02.641782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:02.643640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:02.658249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1172/2365] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:53:02.662786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eaad1-227b-42fd-84e1-947b6c96f42c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:02.673574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006eaad1-227b-42fd-84e1-947b6c96f42c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:02.678750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:02.679467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:02.679860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:02.680141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:02.680898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:02.682966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:02.880977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1194/2366] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:02 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:02.882780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:02.897248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a48af-8584-4a52-ad12-727baf6492d0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:02.969680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:02.969680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:02.969680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:02.969680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:02.970219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:02.972215 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.081980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1173/2367] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:02 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:53:03.083255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:03.093772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9acd3-edef-4144-a132-88b3c5ca960b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:03.098641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:03.099343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.099622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.100087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.100850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:03.102446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.116896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1195/2368] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:03.121985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:03.138316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba766d3a-b8b7-46cf-a996-36194be3c878 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:03.143025 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b038b3-fe86-4014-87ae-d5af859bec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:03.143693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b038b3-fe86-4014-87ae-d5af859bec88 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.143985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b038b3-fe86-4014-87ae-d5af859bec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.144271 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b038b3-fe86-4014-87ae-d5af859bec88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.145040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b038b3-fe86-4014-87ae-d5af859bec88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:03.150384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b038b3-fe86-4014-87ae-d5af859bec88 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.157522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1174/2369] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:03.161833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:03.172043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bab13-96f9-4abc-9404-d540f2950688 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:03.176699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:03.177586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.177873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.178150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] PATH_INFO: `/v3/projects/65cd3ff7501549eca9f717e88909bb54/users/a19632b2b1bc4a1fbdcc9bde563ce628/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.193134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cd3ff7501549eca9f717e88909bb54, user_id=a19632b2b1bc4a1fbdcc9bde563ce628, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:03.195912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.312879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1196/2370] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:03 2022] PUT /identity/v3/projects/65cd3ff7501549eca9f717e88909bb54/users/a19632b2b1bc4a1fbdcc9bde563ce628/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:03.315603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17dac3-2ae0-4e97-9689-9baff4bc7381 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.316309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17dac3-2ae0-4e97-9689-9baff4bc7381 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.316309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17dac3-2ae0-4e97-9689-9baff4bc7381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.340769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:03.353261 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bc17dac3-2ae0-4e97-9689-9baff4bc7381 None None] MFA Rules not processed for user `a19632b2b1bc4a1fbdcc9bde563ce628`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:03.355205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bc17dac3-2ae0-4e97-9689-9baff4bc7381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:03.445832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1175/2371] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:03.446902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:03.531962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace34413-0086-44a7-a608-5002abdadd46 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:03.538854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:03.539210 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.539743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.539743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.540558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:03.542787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.634051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:03.644028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1197/2372] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:03 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:03.645301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17dac3-2ae0-4e97-9689-9baff4bc7381 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:03.657367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17dac3-2ae0-4e97-9689-9baff4bc7381 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:03.719076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:03.719931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.720222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.720506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.721245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:03.724115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.814940 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1176/2373] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:03 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:03.816516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:03.828721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddff06-50a7-443d-b203-96fa70df3219 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:03.836138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:03.836905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.837191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.837469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.838279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:03.839894 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.852705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1198/2374] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:53:03.857228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:03.868320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af4f7c-d913-468a-a347-4759eed08842 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:03.873380 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:03.874293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:03.874579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:03.874857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] PATH_INFO: `/v3/projects/fcbf120c9a724945b075fd532b3f9980/users/65eaf739c69d42b1bc8cdaee542e04a9/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:03.891736 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbf120c9a724945b075fd532b3f9980, user_id=65eaf739c69d42b1bc8cdaee542e04a9, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:03.894517 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:03.986807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1177/2375] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:03 2022] PUT /identity/v3/projects/fcbf120c9a724945b075fd532b3f9980/users/65eaf739c69d42b1bc8cdaee542e04a9/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:03.990859 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:04.004702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28663-29a7-4196-9dd7-d517506ab94b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:04.099860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c80f27-1553-42c2-a638-4f12a20608a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:04.100737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80f27-1553-42c2-a638-4f12a20608a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:04.101100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80f27-1553-42c2-a638-4f12a20608a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:04.101449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80f27-1553-42c2-a638-4f12a20608a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:04.102411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c80f27-1553-42c2-a638-4f12a20608a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:04.104903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c80f27-1553-42c2-a638-4f12a20608a2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:04.118889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1199/2376] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:04.128016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:04.140430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b07f17-567f-44a3-b686-21fce88453f6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:04.146869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:04.147794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:04.148086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:04.148375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] PATH_INFO: `/v3/projects/fcbf120c9a724945b075fd532b3f9980/users/65eaf739c69d42b1bc8cdaee542e04a9/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:04.151133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbf120c9a724945b075fd532b3f9980, user_id=65eaf739c69d42b1bc8cdaee542e04a9, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:04.154586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:04.177922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1178/2377] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:04 2022] PUT /identity/v3/projects/fcbf120c9a724945b075fd532b3f9980/users/65eaf739c69d42b1bc8cdaee542e04a9/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:04.179253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70d64b-e483-441f-a4fb-c7b06d0a7e71 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:04.179551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70d64b-e483-441f-a4fb-c7b06d0a7e71 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:04.179834 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70d64b-e483-441f-a4fb-c7b06d0a7e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:04.208272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ee70d64b-e483-441f-a4fb-c7b06d0a7e71 None None] MFA Rules not processed for user `65eaf739c69d42b1bc8cdaee542e04a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:04.209551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ee70d64b-e483-441f-a4fb-c7b06d0a7e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:04.284909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1200/2378] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:04.300527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:04.300527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:04.388949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895159b1-d85b-4555-ab1d-e3faed1b0b7b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:04.392050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6c52d-4043-4de4-b88f-4b4338b09cd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:04.392756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c52d-4043-4de4-b88f-4b4338b09cd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:04.393026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c52d-4043-4de4-b88f-4b4338b09cd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:04.393305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c52d-4043-4de4-b88f-4b4338b09cd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:04.401627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:04.406159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6c52d-4043-4de4-b88f-4b4338b09cd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:04.407760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6c52d-4043-4de4-b88f-4b4338b09cd4 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:04.422035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1179/2379] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:04.491293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70d64b-e483-441f-a4fb-c7b06d0a7e71 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:04.562369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70d64b-e483-441f-a4fb-c7b06d0a7e71 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:04.566968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:04.567844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:04.568133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:04.568456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] PATH_INFO: `/v3/users/a19632b2b1bc4a1fbdcc9bde563ce628` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:04.569773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19632b2b1bc4a1fbdcc9bde563ce628)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:04.571705 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:04.586400 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:04.933030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19632b2b1bc4a1fbdcc9bde563ce628'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:04.951869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19632b2b1bc4a1fbdcc9bde563ce628'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:04.953210 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19632b2b1bc4a1fbdcc9bde563ce628'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:04.962021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19632b2b1bc4a1fbdcc9bde563ce628'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:04.963534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19632b2b1bc4a1fbdcc9bde563ce628'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:05.072748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1201/2380] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:04 2022] DELETE /identity/v3/users/a19632b2b1bc4a1fbdcc9bde563ce628 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:05.077258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:05.077752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:05.077869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:05.096152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:05.097404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:05.157318 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1180/2381] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:05.167617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:05.169162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:05.233492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fffc7-9685-43e3-8f9d-87f960da7ba8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:05.238617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:05.239277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:05.239585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:05.239858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:05.250135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:05.252097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:05.263120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1202/2382] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:05.672318 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:05.673794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:05.686752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4ad8e-e880-4b04-823d-b9b30c5a3c03 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:05.691376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:05.692359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:05.692527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:05.692842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] PATH_INFO: `/v3/projects/65cd3ff7501549eca9f717e88909bb54` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:05.694069 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cd3ff7501549eca9f717e88909bb54)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:05.696000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:05.798311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cd3ff7501549eca9f717e88909bb54', 'request_id': 'req-bc5eabee-df27-42d5-b4ff-da83e0815b41'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:05.804471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cd3ff7501549eca9f717e88909bb54', 'request_id': 'req-bc5eabee-df27-42d5-b4ff-da83e0815b41'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:05.883578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cd3ff7501549eca9f717e88909bb54', 'request_id': 'req-bc5eabee-df27-42d5-b4ff-da83e0815b41'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:05.898306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] The token cache is being invalidate because project 65cd3ff7501549eca9f717e88909bb54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:05.899221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:05.899701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:05.902013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1181/2383] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:05 2022] DELETE /identity/v3/projects/65cd3ff7501549eca9f717e88909bb54 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:05.907144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:05.918280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20f817-c57b-4dd0-ad44-5d035d0ada6e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:05.974247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:05.975123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:05.975399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:05.975671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] PATH_INFO: `/v3/users/65eaf739c69d42b1bc8cdaee542e04a9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:05.976895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65eaf739c69d42b1bc8cdaee542e04a9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:05.979559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:06.157842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:06.167341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65eaf739c69d42b1bc8cdaee542e04a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.179187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65eaf739c69d42b1bc8cdaee542e04a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.179795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65eaf739c69d42b1bc8cdaee542e04a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.186723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65eaf739c69d42b1bc8cdaee542e04a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.187362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-cb0ee519-0446-4c74-9de9-07ddc0800266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65eaf739c69d42b1bc8cdaee542e04a9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.278742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1203/2384] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:05 2022] DELETE /identity/v3/users/65eaf739c69d42b1bc8cdaee542e04a9 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:06.662111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:06.663555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:06.679451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5eabee-df27-42d5-b4ff-da83e0815b41 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:06.743615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:06.744402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:06.744735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:06.745014 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] PATH_INFO: `/v3/projects/fcbf120c9a724945b075fd532b3f9980` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:06.746197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbf120c9a724945b075fd532b3f9980)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:06.748134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:06.769052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:06.867346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbf120c9a724945b075fd532b3f9980', 'request_id': 'req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.873602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbf120c9a724945b075fd532b3f9980', 'request_id': 'req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.958037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbf120c9a724945b075fd532b3f9980', 'request_id': 'req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.971419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] The token cache is being invalidate because project fcbf120c9a724945b075fd532b3f9980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:06.971713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.972363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-694c78a9-0a1a-45f2-8e50-c9b71fbaca95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:06.973955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1182/2385] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:06 2022] DELETE /identity/v3/projects/fcbf120c9a724945b075fd532b3f9980 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:06.979538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e381-652f-4874-82e4-84f09ef4b63d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:06.982884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e381-652f-4874-82e4-84f09ef4b63d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:06.982884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e381-652f-4874-82e4-84f09ef4b63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:06.998727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d6a6e381-652f-4874-82e4-84f09ef4b63d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:06.999868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d6a6e381-652f-4874-82e4-84f09ef4b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:07.060315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1204/2386] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:07.069211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3b002-2262-4b7a-9807-5e09a9d13c85 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.069503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3b002-2262-4b7a-9807-5e09a9d13c85 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.069780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3b002-2262-4b7a-9807-5e09a9d13c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.088864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8ea3b002-2262-4b7a-9807-5e09a9d13c85 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:07.090055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8ea3b002-2262-4b7a-9807-5e09a9d13c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:07.097842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1183/2387] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:07.101959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6e381-652f-4874-82e4-84f09ef4b63d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.111931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6e381-652f-4874-82e4-84f09ef4b63d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.116695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.117345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.117630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.117907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.118634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:53:07.118940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.120541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:07.132470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1205/2388] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:53:07.137335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3b002-2262-4b7a-9807-5e09a9d13c85 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.146701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3b002-2262-4b7a-9807-5e09a9d13c85 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.152209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.153155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.153440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.153725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.154426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.156367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:07.245753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1184/2389] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:07 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:07.249770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.261711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45543718-d5df-41eb-8b7f-cd0d486f59c0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.327026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.328092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.328092 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.328283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.329049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.331013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:07.415477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1206/2390] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:07 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:07.419554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.429908 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df009d40-a40d-4011-bc45-f378d5353370 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.434835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.435502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.435792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.436076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.436878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.438536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:07.449525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1185/2391] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:07.453476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.464239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036a96d-31f8-4f9c-9dfe-9c11d1275c51 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.468997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5348489-6ff1-4cd5-a7a0-9a4f3051934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.469652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5348489-6ff1-4cd5-a7a0-9a4f3051934e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.469940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5348489-6ff1-4cd5-a7a0-9a4f3051934e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.470218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5348489-6ff1-4cd5-a7a0-9a4f3051934e admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.470944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5348489-6ff1-4cd5-a7a0-9a4f3051934e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.472541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5348489-6ff1-4cd5-a7a0-9a4f3051934e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:07.482733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1207/2392] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:53:07.486801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.497397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e059c3f-8b59-4b27-8b9b-cf39b8f03f60 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.502287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.503149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.504956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.509801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] PATH_INFO: `/v3/projects/ad54253c513f43fbac984eba88b25ad0/users/9fa3079439304c80939397e6a30982f9/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.521358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad54253c513f43fbac984eba88b25ad0, user_id=9fa3079439304c80939397e6a30982f9, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.524105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:07.611886 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1186/2393] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:07 2022] PUT /identity/v3/projects/ad54253c513f43fbac984eba88b25ad0/users/9fa3079439304c80939397e6a30982f9/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:53:07.623184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56e727-46a3-4892-b952-5062bc0ce986 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.623480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56e727-46a3-4892-b952-5062bc0ce986 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.623763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56e727-46a3-4892-b952-5062bc0ce986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.655437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2a56e727-46a3-4892-b952-5062bc0ce986 None None] MFA Rules not processed for user `9fa3079439304c80939397e6a30982f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:07.656762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2a56e727-46a3-4892-b952-5062bc0ce986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:07.661936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:07.724341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1208/2394] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:07.728572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.790913 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f481c5-a361-4d69-ac2d-83cbbb36f58d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.796179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.796868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.797156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.797437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.798121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.800054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:07.916448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1187/2395] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:07 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:07.917859 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56e727-46a3-4892-b952-5062bc0ce986 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:07.931670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56e727-46a3-4892-b952-5062bc0ce986 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:07.990795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:07.991477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:07.991769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:07.992062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:07.992801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:07.994777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.070735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1209/2396] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:07 2022] POST /identity/v3/users => generated 294 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:53:08.073165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.083295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1831003-117d-4e29-b1cb-d134742a8772 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.087899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.089153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.089460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.089718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.090455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.092081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.103747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1188/2397] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:08.107626 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.117805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec35d-f20b-46a9-8207-b80a2d3c879c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.122461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8daa1-65a0-4d69-99e4-70feffae0fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.123126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8daa1-65a0-4d69-99e4-70feffae0fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.123417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8daa1-65a0-4d69-99e4-70feffae0fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.123699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8daa1-65a0-4d69-99e4-70feffae0fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.124429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8daa1-65a0-4d69-99e4-70feffae0fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.126333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8daa1-65a0-4d69-99e4-70feffae0fa7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.139936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1210/2398] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:53:08.141238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.151686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a38aca-60df-418e-b3ea-a36198e426f9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.156547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.157458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.157746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.158031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] PATH_INFO: `/v3/projects/78f90b7581f643a4b32bddc1663d41a0/users/126ce213e226414997946e2ba3a31734/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.173028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f90b7581f643a4b32bddc1663d41a0, user_id=126ce213e226414997946e2ba3a31734, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.175730 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.177961 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:08.290719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1189/2399] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:08 2022] PUT /identity/v3/projects/78f90b7581f643a4b32bddc1663d41a0/users/126ce213e226414997946e2ba3a31734/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:08.301209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef8de7-5701-48d3-b5d1-91778573d176 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.302951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef8de7-5701-48d3-b5d1-91778573d176 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.302951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef8de7-5701-48d3-b5d1-91778573d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.333143 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-deef8de7-5701-48d3-b5d1-91778573d176 None None] MFA Rules not processed for user `126ce213e226414997946e2ba3a31734`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:08.335172 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-deef8de7-5701-48d3-b5d1-91778573d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:08.402575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1211/2400] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:08.406649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.471048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4693be9-b547-4be6-b8ba-fd9642a84deb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.476267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.476973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.477251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.477535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.478248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.480186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.565267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1190/2401] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:08 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:08.569457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef8de7-5701-48d3-b5d1-91778573d176 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.581662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef8de7-5701-48d3-b5d1-91778573d176 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.595225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:08.643469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.644170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.644454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.644781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.645469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.647563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.749913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1212/2402] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:08 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:08.751588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.762616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a85d64-ee1f-45c0-9836-266538078b6e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.767482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.768171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.768472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.768772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.769502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.771122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.787246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1191/2403] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:08.788175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.798573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd75262-461b-4d8f-9568-1d4d313c1c8f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.803239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.804118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.804441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.804743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] PATH_INFO: `/v3/projects/15c5ea0267dd4542b1a68046cfdfd70b/users/0f95810c828447c1bbdb75b5c6b6fc7f/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.820500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c5ea0267dd4542b1a68046cfdfd70b, user_id=0f95810c828447c1bbdb75b5c6b6fc7f, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.824064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.890096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1213/2404] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:08 2022] PUT /identity/v3/projects/15c5ea0267dd4542b1a68046cfdfd70b/users/0f95810c828447c1bbdb75b5c6b6fc7f/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:08.893968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:08.904526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ba949-e1ca-4a46-8f12-15ca2eea99b1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:08.969334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752faa28-2c91-4bc4-b29b-04b9051d90bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:08.970248 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752faa28-2c91-4bc4-b29b-04b9051d90bb admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:08.970540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752faa28-2c91-4bc4-b29b-04b9051d90bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:08.970822 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752faa28-2c91-4bc4-b29b-04b9051d90bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:08.971831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752faa28-2c91-4bc4-b29b-04b9051d90bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:08.973504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752faa28-2c91-4bc4-b29b-04b9051d90bb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:08.985207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1192/2405] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:08.989395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:09.000684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9d79c-8375-4976-b2f0-736a76042941 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:09.005474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:09.006354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:09.006643 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:09.006926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] PATH_INFO: `/v3/projects/15c5ea0267dd4542b1a68046cfdfd70b/users/0f95810c828447c1bbdb75b5c6b6fc7f/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:09.009017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c5ea0267dd4542b1a68046cfdfd70b, user_id=0f95810c828447c1bbdb75b5c6b6fc7f, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:09.012255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:09.022690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1214/2406] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:08 2022] PUT /identity/v3/projects/15c5ea0267dd4542b1a68046cfdfd70b/users/0f95810c828447c1bbdb75b5c6b6fc7f/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:09.033770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:09.034152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:09.034533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:09.060807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] MFA Rules not processed for user `0f95810c828447c1bbdb75b5c6b6fc7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:09.062010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:09.110586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:09.129304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1193/2407] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:09.140667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:09.145412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:09.200567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a3fdb-97ee-4d08-ba22-fa4afc0384a2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:09.204468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:09.205152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:09.205441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:09.205784 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:09.215818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:09.219967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:09.227412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1215/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:10.187897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:10.189555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:10.204475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf062f-5470-4732-8b8b-75e72ee4eea1 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:10.209957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:10.209957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:10.209957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:10.209957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:10.276497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:10.278633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:10.287209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:10.288782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:10.298122 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1194/2409] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:10.354797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e393-fcbf-4e8f-aabc-77b516d7133f service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:10.360523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01301b-65a2-4f80-9dc0-21ba06185e0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:10.361310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01301b-65a2-4f80-9dc0-21ba06185e0a service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:10.361652 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01301b-65a2-4f80-9dc0-21ba06185e0a service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:10.361938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01301b-65a2-4f80-9dc0-21ba06185e0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:10.363741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5e01301b-65a2-4f80-9dc0-21ba06185e0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:10.374237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:10.377006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01301b-65a2-4f80-9dc0-21ba06185e0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:10.378621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01301b-65a2-4f80-9dc0-21ba06185e0a service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:10.391614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1216/2410] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:53:15.249618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:15.277264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24d054-9503-4ebe-abfc-82443fc8ddf9 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:15.281923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:15.282974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:15.282974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:15.283231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:15.387377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:15.389218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:15.413916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1195/2411] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:15.921459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc23b-798b-4af7-a5d6-08afb3a25593 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:15.921971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc23b-798b-4af7-a5d6-08afb3a25593 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:15.922090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc23b-798b-4af7-a5d6-08afb3a25593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:15.941001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-94bcc23b-798b-4af7-a5d6-08afb3a25593 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:15.942222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-94bcc23b-798b-4af7-a5d6-08afb3a25593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:15.999149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1217/2412] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:53:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:16.128649 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:16.130099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:16.218458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015a0cf-8910-478d-be43-a496fffd002d service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:16.224686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1011f-fc42-4eed-a1d6-c4a7989420c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:16.225363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1011f-fc42-4eed-a1d6-c4a7989420c5 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:16.225654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1011f-fc42-4eed-a1d6-c4a7989420c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:16.225947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1011f-fc42-4eed-a1d6-c4a7989420c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:16.236465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1011f-fc42-4eed-a1d6-c4a7989420c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:16.238212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1011f-fc42-4eed-a1d6-c4a7989420c5 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:16.251176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1196/2413] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:53:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:53:19.729754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc23b-798b-4af7-a5d6-08afb3a25593 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:19.809850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc23b-798b-4af7-a5d6-08afb3a25593 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:19.820685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:19.824144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:19.824734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:19.825018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] PATH_INFO: `/v3/users/9fa3079439304c80939397e6a30982f9` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:19.826360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa3079439304c80939397e6a30982f9)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:19.828337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:20.025184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa3079439304c80939397e6a30982f9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:20.043002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa3079439304c80939397e6a30982f9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:20.044034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa3079439304c80939397e6a30982f9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:20.051904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa3079439304c80939397e6a30982f9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:20.053158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa3079439304c80939397e6a30982f9'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:20.135366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1218/2414] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:19 2022] DELETE /identity/v3/users/9fa3079439304c80939397e6a30982f9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:20.140162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:20.140723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:20.140821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:20.159054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:20.160758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:20.171739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:20.239988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1197/2415] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:20.252580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:20.254087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:20.278530 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:20.325203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9a31-e9d7-498c-9370-2a962c0e9656 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:20.328911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:20.329565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:20.329851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:20.330124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:20.345448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:20.347644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:20.359898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1219/2416] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:20.816997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:20.818660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:20.961401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917d74f-16f7-42bf-b89a-2e8db446c0af None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:20.970982 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:20.971875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:20.972160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:20.972437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] PATH_INFO: `/v3/projects/ad54253c513f43fbac984eba88b25ad0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:20.973793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad54253c513f43fbac984eba88b25ad0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:20.975776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:21.079958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad54253c513f43fbac984eba88b25ad0', 'request_id': 'req-b107a824-5fab-4fce-9906-0f4611c665c4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.086895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad54253c513f43fbac984eba88b25ad0', 'request_id': 'req-b107a824-5fab-4fce-9906-0f4611c665c4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.146614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad54253c513f43fbac984eba88b25ad0', 'request_id': 'req-b107a824-5fab-4fce-9906-0f4611c665c4'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.179037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] The token cache is being invalidate because project ad54253c513f43fbac984eba88b25ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:21.179334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.180336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.182485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1198/2417] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:20 2022] DELETE /identity/v3/projects/ad54253c513f43fbac984eba88b25ad0 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:21.188244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:21.259765 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56cc8b-3ed2-4b77-ba01-d893e9583ee6 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:21.340376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:21.341207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:21.341496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:21.341775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] PATH_INFO: `/v3/users/126ce213e226414997946e2ba3a31734` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:21.343169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126ce213e226414997946e2ba3a31734)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:21.345340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:21.520708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126ce213e226414997946e2ba3a31734'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.535045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126ce213e226414997946e2ba3a31734'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.535811 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126ce213e226414997946e2ba3a31734'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.545133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126ce213e226414997946e2ba3a31734'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.545877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126ce213e226414997946e2ba3a31734'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:21.619398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1220/2418] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:21 2022] DELETE /identity/v3/users/126ce213e226414997946e2ba3a31734 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:22.170868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:22.172379 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:22.186114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107a824-5fab-4fce-9906-0f4611c665c4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:22.235543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:22.268612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:22.269693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:22.270009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:22.270335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] PATH_INFO: `/v3/projects/78f90b7581f643a4b32bddc1663d41a0` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:22.272093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f90b7581f643a4b32bddc1663d41a0)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:22.274715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:22.420629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f90b7581f643a4b32bddc1663d41a0', 'request_id': 'req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.432392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f90b7581f643a4b32bddc1663d41a0', 'request_id': 'req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.506462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f90b7581f643a4b32bddc1663d41a0', 'request_id': 'req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.521890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] The token cache is being invalidate because project 78f90b7581f643a4b32bddc1663d41a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:22.522459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.523395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.526078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1199/2419] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:22 2022] DELETE /identity/v3/projects/78f90b7581f643a4b32bddc1663d41a0 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:22.532267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:22.533722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:22.550927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52ab32-d956-4dd7-b479-a12578649b20 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:22.557380 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:22.621100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:22.622149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:22.622494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:22.622868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] PATH_INFO: `/v3/users/0f95810c828447c1bbdb75b5c6b6fc7f` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:22.624347 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f95810c828447c1bbdb75b5c6b6fc7f)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:22.626729 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:22.830897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f95810c828447c1bbdb75b5c6b6fc7f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.857189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f95810c828447c1bbdb75b5c6b6fc7f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.858154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f95810c828447c1bbdb75b5c6b6fc7f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.869000 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f95810c828447c1bbdb75b5c6b6fc7f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.870011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-4a8690d3-264c-4c07-8daa-fd23ac37cdd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f95810c828447c1bbdb75b5c6b6fc7f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:22.942593 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1221/2420] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:22 2022] DELETE /identity/v3/users/0f95810c828447c1bbdb75b5c6b6fc7f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:23.401978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:23.403506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:23.443613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a17fef-4272-4ffc-86a7-e03a2b2f3ac9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:23.524550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:23.524550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:23.524844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:23.525989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] PATH_INFO: `/v3/projects/15c5ea0267dd4542b1a68046cfdfd70b` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:23.529574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c5ea0267dd4542b1a68046cfdfd70b)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:23.532021 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:23.673844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c5ea0267dd4542b1a68046cfdfd70b', 'request_id': 'req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:23.685034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c5ea0267dd4542b1a68046cfdfd70b', 'request_id': 'req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:23.735574 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c5ea0267dd4542b1a68046cfdfd70b', 'request_id': 'req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:23.751738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] The token cache is being invalidate because project 15c5ea0267dd4542b1a68046cfdfd70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:23.752342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:23.753268 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-462b4bfc-7b75-46cd-a8bd-96f30e9cf38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:23.754037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:23.756538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1200/2421] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:23 2022] DELETE /identity/v3/projects/15c5ea0267dd4542b1a68046cfdfd70b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:23.763749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca4fae-e93b-421d-b6f2-b41b506386a1 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:23.764690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca4fae-e93b-421d-b6f2-b41b506386a1 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:23.764690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca4fae-e93b-421d-b6f2-b41b506386a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:23.841976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-02ca4fae-e93b-421d-b6f2-b41b506386a1 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:23.844074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-02ca4fae-e93b-421d-b6f2-b41b506386a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:23.907342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:23.922450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1222/2422] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:23.931269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54317aad-5004-47fd-9fab-5286d16637d8 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:23.932235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54317aad-5004-47fd-9fab-5286d16637d8 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:23.932583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54317aad-5004-47fd-9fab-5286d16637d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:23.957202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-54317aad-5004-47fd-9fab-5286d16637d8 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:23.958562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-54317aad-5004-47fd-9fab-5286d16637d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:23.980902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1201/2423] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:23.985481 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca4fae-e93b-421d-b6f2-b41b506386a1 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:23.998247 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca4fae-e93b-421d-b6f2-b41b506386a1 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:24.005283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:24.006249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.006608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.006957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.007844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:53:24.008237 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:24.010387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:24.028653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1223/2424] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:53:24.031183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54317aad-5004-47fd-9fab-5286d16637d8 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:24.044378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54317aad-5004-47fd-9fab-5286d16637d8 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:24.049708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:24.050378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.050668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.050946 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.052645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:24.053927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:24.163550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1202/2425] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:24 2022] POST /identity/v3/projects => generated 391 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:24.168122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:24.182066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf9e3b-ed6b-4546-8518-4ac1de62b514 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:24.257988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:24.258818 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.259261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.259640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.260443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:24.262544 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:24.353914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1224/2426] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:24 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:53:24.357988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:24.370033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5371f-050c-48cd-ae00-d99c48f3f053 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:24.378448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:24.379338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.379718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.380078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.380887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:24.382529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:24.394278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1203/2427] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:24.398173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:24.416585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38657f3d-83ee-4c59-bbca-138e7996d7f8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:24.422546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aca75c-8dc6-416d-b890-e5f01fdcdd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:24.423739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca75c-8dc6-416d-b890-e5f01fdcdd62 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.424276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca75c-8dc6-416d-b890-e5f01fdcdd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.424889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aca75c-8dc6-416d-b890-e5f01fdcdd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.426159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aca75c-8dc6-416d-b890-e5f01fdcdd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:24.428178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aca75c-8dc6-416d-b890-e5f01fdcdd62 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:24.443473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1225/2428] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:24.445280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:24.460468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ea5d0-50db-4f56-8e9b-f027b511cd52 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:24.467162 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:24.468153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.468512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.468892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] PATH_INFO: `/v3/projects/18a2e177b19e4f2db5e8cf4bb083dca7/users/4cd77bcb11714dbba7ec8df355340150/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.492506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a2e177b19e4f2db5e8cf4bb083dca7, user_id=4cd77bcb11714dbba7ec8df355340150, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:24.495856 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:24.627592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1204/2429] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:24 2022] PUT /identity/v3/projects/18a2e177b19e4f2db5e8cf4bb083dca7/users/4cd77bcb11714dbba7ec8df355340150/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:24.641262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bb88d-c216-4418-bde0-0aa47abda2e9 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.641576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bb88d-c216-4418-bde0-0aa47abda2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.641864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bb88d-c216-4418-bde0-0aa47abda2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.673532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cd3bb88d-c216-4418-bde0-0aa47abda2e9 None None] MFA Rules not processed for user `4cd77bcb11714dbba7ec8df355340150`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:24.674944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cd3bb88d-c216-4418-bde0-0aa47abda2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:24.762419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1226/2430] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:24.767660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:24.855592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f16beb-6c49-4289-806f-6f757f475987 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:24.866245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:24.867139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:24.867491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:24.868106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:24.869571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:24.872318 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.011076 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1205/2431] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:24 2022] POST /identity/v3/projects => generated 391 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:53:25.015051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bb88d-c216-4418-bde0-0aa47abda2e9 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:25.026575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bb88d-c216-4418-bde0-0aa47abda2e9 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:25.093330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:25.113433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:25.114898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.114898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.114898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.115331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:25.117331 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.207853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1227/2432] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:25 2022] POST /identity/v3/users => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:25.211527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:25.229466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a68124-47a6-4998-964a-9641c7815ceb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:25.234611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:25.235437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.235980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.236383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.237407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:25.239320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.256587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1206/2433] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:25.261047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:25.273945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f50d3-df84-4d27-abf4-1ff0e6ace45f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:25.279591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:25.280486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.280823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.281107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] PATH_INFO: `/v3/projects/a079fdb1cb114217a0e18bbac63a28fd/users/acbd809f62e84218a1b9c19e5cefecbd/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.300295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a079fdb1cb114217a0e18bbac63a28fd, user_id=acbd809f62e84218a1b9c19e5cefecbd, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:25.303224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.408914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1228/2434] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:25 2022] PUT /identity/v3/projects/a079fdb1cb114217a0e18bbac63a28fd/users/acbd809f62e84218a1b9c19e5cefecbd/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:25.409998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:25.426477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0da56-ce84-4774-bb58-a52c15b3cc65 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:25.497869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9027f5-9642-4470-bfd3-2561e8c3df79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:25.497869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9027f5-9642-4470-bfd3-2561e8c3df79 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.497869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9027f5-9642-4470-bfd3-2561e8c3df79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.497869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9027f5-9642-4470-bfd3-2561e8c3df79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.499077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9027f5-9642-4470-bfd3-2561e8c3df79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:25.499615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9027f5-9642-4470-bfd3-2561e8c3df79 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.501415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:25.512638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1207/2435] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:53:25.515529 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:25.527356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ec5f-0de7-431b-b6a0-33fe14818b63 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:25.533524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:25.534519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.534898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.535179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] PATH_INFO: `/v3/projects/a079fdb1cb114217a0e18bbac63a28fd/users/acbd809f62e84218a1b9c19e5cefecbd/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.537210 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a079fdb1cb114217a0e18bbac63a28fd, user_id=acbd809f62e84218a1b9c19e5cefecbd, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:25.539951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.553065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1229/2436] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:25 2022] PUT /identity/v3/projects/a079fdb1cb114217a0e18bbac63a28fd/users/acbd809f62e84218a1b9c19e5cefecbd/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:25.563374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.563673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.563977 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.592207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] MFA Rules not processed for user `acbd809f62e84218a1b9c19e5cefecbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:25.593740 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:25.674055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1208/2437] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:25.686750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:25.688487 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:25.789931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992019b-5b50-4f8c-905b-2d42c31dedea admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:25.794224 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:25.794880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.795169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.795448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.805040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:25.806490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.816155 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1230/2438] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:53:25.861918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:25.863407 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:25.875684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a13cca-e906-4fde-9f4e-24c9d57f4a9e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:25.879312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd801a5-39bf-4733-bde3-cb85cd6cd1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:25.879958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd801a5-39bf-4733-bde3-cb85cd6cd1df service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:25.880247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd801a5-39bf-4733-bde3-cb85cd6cd1df service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:25.880530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd801a5-39bf-4733-bde3-cb85cd6cd1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:25.937089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd801a5-39bf-4733-bde3-cb85cd6cd1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:25.938545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd801a5-39bf-4733-bde3-cb85cd6cd1df service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:25.947960 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1209/2439] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:30.621971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:30.686759 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212f4b-5a4b-424f-9e2f-0b3b9de93595 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:30.692253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:30.693158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:30.693461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:30.693724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] PATH_INFO: `/v3/users/4cd77bcb11714dbba7ec8df355340150` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:30.695033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd77bcb11714dbba7ec8df355340150)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:30.697698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:30.712777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:30.897644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd77bcb11714dbba7ec8df355340150'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:30.911619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd77bcb11714dbba7ec8df355340150'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:30.912346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd77bcb11714dbba7ec8df355340150'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:30.919858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd77bcb11714dbba7ec8df355340150'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:30.920637 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd77bcb11714dbba7ec8df355340150'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:30.968547 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1231/2440] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:30 2022] DELETE /identity/v3/users/4cd77bcb11714dbba7ec8df355340150 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:30.973371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:30.973848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:30.973932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:30.991133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:30.992343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:31.052512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1210/2441] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:31.062969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:31.064509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:31.144495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70804f1-05e4-4c21-bc79-2316aca4aecc admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:31.148365 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:31.149071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:31.149434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:31.149715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:31.159891 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:31.161349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:31.171694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1232/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:31.633078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:31.634532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:31.646623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968bb4b-771a-4d0f-8764-5152f0946a46 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:31.651884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:31.652800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:31.653121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:31.653402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] PATH_INFO: `/v3/projects/18a2e177b19e4f2db5e8cf4bb083dca7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:31.654586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a2e177b19e4f2db5e8cf4bb083dca7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:31.656505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:31.672951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:31.763341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a2e177b19e4f2db5e8cf4bb083dca7', 'request_id': 'req-057374dc-e741-4cc6-98f9-97c360255a44'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:31.769483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a2e177b19e4f2db5e8cf4bb083dca7', 'request_id': 'req-057374dc-e741-4cc6-98f9-97c360255a44'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:31.831267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a2e177b19e4f2db5e8cf4bb083dca7', 'request_id': 'req-057374dc-e741-4cc6-98f9-97c360255a44'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:31.842712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] The token cache is being invalidate because project 18a2e177b19e4f2db5e8cf4bb083dca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:31.843029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:31.843637 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:31.845189 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1211/2443] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:31 2022] DELETE /identity/v3/projects/18a2e177b19e4f2db5e8cf4bb083dca7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:31.849254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:31.859090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dba454-01d9-4bb3-aa9c-2024aae2806a service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:31.915057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:31.915881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:31.916167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:31.916446 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] PATH_INFO: `/v3/users/acbd809f62e84218a1b9c19e5cefecbd` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:31.917897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acbd809f62e84218a1b9c19e5cefecbd)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:31.920212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:31.935966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:32.115782 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acbd809f62e84218a1b9c19e5cefecbd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.127772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acbd809f62e84218a1b9c19e5cefecbd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.128433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acbd809f62e84218a1b9c19e5cefecbd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.135768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acbd809f62e84218a1b9c19e5cefecbd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.136385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-e5c52954-6920-4f2e-9ef7-9628024d8713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acbd809f62e84218a1b9c19e5cefecbd'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.220007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1233/2444] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:31 2022] DELETE /identity/v3/users/acbd809f62e84218a1b9c19e5cefecbd => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:32.611040 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:32.612805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:32.629392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057374dc-e741-4cc6-98f9-97c360255a44 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:32.654735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d479e4-4be1-49bf-b60f-fe4582818f5c None None] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:32.655027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d479e4-4be1-49bf-b60f-fe4582818f5c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:32.655306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d479e4-4be1-49bf-b60f-fe4582818f5c None None] PATH_INFO: `/` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:32.656734 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1234/2445] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:53:32.662843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:32.663215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:32.663568 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:32.693657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:32.695398 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:32.705345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:32.706231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:32.706522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:32.706807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] PATH_INFO: `/v3/projects/a079fdb1cb114217a0e18bbac63a28fd` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:32.708555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a079fdb1cb114217a0e18bbac63a28fd)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:32.710560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:32.775663 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1235/2446] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:32.787010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:32.788514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:32.843931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a079fdb1cb114217a0e18bbac63a28fd', 'request_id': 'req-c2f22e6c-460d-4417-8044-4cec845fd19f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.850704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a079fdb1cb114217a0e18bbac63a28fd', 'request_id': 'req-c2f22e6c-460d-4417-8044-4cec845fd19f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.851290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a310a-0eba-46e1-a380-a7914153c712 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:32.886770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:32.907775 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a2a0e-f22c-4384-af6c-043c8b8f8977 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:32.908585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2a0e-f22c-4384-af6c-043c8b8f8977 service glance] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:32.908925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2a0e-f22c-4384-af6c-043c8b8f8977 service glance] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:32.909207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2a0e-f22c-4384-af6c-043c8b8f8977 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:32.941027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a079fdb1cb114217a0e18bbac63a28fd', 'request_id': 'req-c2f22e6c-460d-4417-8044-4cec845fd19f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.955933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] The token cache is being invalidate because project a079fdb1cb114217a0e18bbac63a28fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:32.956377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.957045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c2f22e6c-460d-4417-8044-4cec845fd19f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:32.958806 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1212/2447] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:32 2022] DELETE /identity/v3/projects/a079fdb1cb114217a0e18bbac63a28fd => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:53:32.964242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a0b3-3865-4e35-94d5-42f3705bca96 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:32.964539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a0b3-3865-4e35-94d5-42f3705bca96 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:32.964878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a0b3-3865-4e35-94d5-42f3705bca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:32.987657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a2a0e-f22c-4384-af6c-043c8b8f8977 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:32.989117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a2a0e-f22c-4384-af6c-043c8b8f8977 service glance] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:32.991319 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c1b7a0b3-3865-4e35-94d5-42f3705bca96 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:32.994953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c1b7a0b3-3865-4e35-94d5-42f3705bca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:33.001122 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1236/2448] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:53:33.027762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:33.057289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1213/2449] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:33.065231 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5688a9-9be5-425a-ac68-c9d8fb4e11c4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.065608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5688a9-9be5-425a-ac68-c9d8fb4e11c4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.065961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5688a9-9be5-425a-ac68-c9d8fb4e11c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.085622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dd5688a9-9be5-425a-ac68-c9d8fb4e11c4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:33.086846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dd5688a9-9be5-425a-ac68-c9d8fb4e11c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:33.095169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1237/2450] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:33.099593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a0b3-3865-4e35-94d5-42f3705bca96 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.109232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a0b3-3865-4e35-94d5-42f3705bca96 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.113825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6d568-1ed6-4f1d-b6e5-c4a5733a4d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.114477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6d568-1ed6-4f1d-b6e5-c4a5733a4d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.114766 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6d568-1ed6-4f1d-b6e5-c4a5733a4d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.115049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6d568-1ed6-4f1d-b6e5-c4a5733a4d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.115834 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6d568-1ed6-4f1d-b6e5-c4a5733a4d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:53:33.116149 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6d568-1ed6-4f1d-b6e5-c4a5733a4d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.117840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6d568-1ed6-4f1d-b6e5-c4a5733a4d4a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.130338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1214/2451] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:53:33.135538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5688a9-9be5-425a-ac68-c9d8fb4e11c4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.146251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5688a9-9be5-425a-ac68-c9d8fb4e11c4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.151118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.151786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.152070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.152346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.153089 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.155054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.211127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7497d-a0e8-47f6-903d-fbf324c55367 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.211429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7497d-a0e8-47f6-903d-fbf324c55367 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.211691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7497d-a0e8-47f6-903d-fbf324c55367 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.213192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1215/2452] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:53:33.217691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.217997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.218280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.233798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1238/2453] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:33 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:53:33.242108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] MFA Rules not processed for user `35d3862fe9e846f18d8bc3a0fa5e993a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:33.242254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.242363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:33.296792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1216/2454] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:33.305645 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a2f22-8cda-4c32-9ad3-f5fb653e20c1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.311109 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.311373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.311791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.312147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.312431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.312967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:33.313169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.315311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.325976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f59f1-5bbd-41ed-ae2c-f15b622996cf None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.330633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72356f-d8f3-4012-b022-4e39674d3a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.331417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72356f-d8f3-4012-b022-4e39674d3a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.331702 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72356f-d8f3-4012-b022-4e39674d3a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.332005 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72356f-d8f3-4012-b022-4e39674d3a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.340967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72356f-d8f3-4012-b022-4e39674d3a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.342747 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72356f-d8f3-4012-b022-4e39674d3a92 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.354232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1217/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:53:33.450522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aa9c6-f571-4290-aa28-9f575b735fd2 None None] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.450836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aa9c6-f571-4290-aa28-9f575b735fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.451128 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aa9c6-f571-4290-aa28-9f575b735fd2 None None] PATH_INFO: `/` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.452793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1218/2456] 10.222.0.57 () {54 vars in 921 bytes} [Fri Jul 15 17:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 17:53:33.456985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.457266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.457545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.476454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] MFA Rules not processed for user `3ba11eb5837a4567bd77f3fcb0017ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:33.478205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:33.509893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1239/2457] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:33 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:53:33.511174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.522604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a5501-dc5e-4555-95f6-a4131ca54eaa admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.527612 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.528280 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.528566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.528941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.529678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.531349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.542512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1240/2458] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:53:33.546363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.551849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1219/2459] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Jul 15 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:33.562872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36f9ab-18b5-481f-82d6-531e2649d3e7 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.563489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.564168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.564456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.564767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.565507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.567447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.568682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.569058 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:33.580957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1241/2460] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:53:33.582585 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdfc60-a00f-471f-bbcc-06f5b0e13a50 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.585312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.588066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93bfc6-5f79-4312-a4a9-ff99c67d8221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.588066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93bfc6-5f79-4312-a4a9-ff99c67d8221 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.588066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93bfc6-5f79-4312-a4a9-ff99c67d8221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.588066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93bfc6-5f79-4312-a4a9-ff99c67d8221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.596395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c38fa-1ec3-42fd-8fd5-f5c9c290d203 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.599015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93bfc6-5f79-4312-a4a9-ff99c67d8221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.600528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93bfc6-5f79-4312-a4a9-ff99c67d8221 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.601479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.602432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.603334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.603677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] PATH_INFO: `/v3/projects/45a2fab56e354f4fac76ad775d3f39c5/users/88c0001a6d384e39b132bb0319eeef9b/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.612470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1220/2461] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:53:33.626244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:33.626244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a2fab56e354f4fac76ad775d3f39c5, user_id=88c0001a6d384e39b132bb0319eeef9b, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.626244 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:33.719991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1242/2462] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:33 2022] PUT /identity/v3/projects/45a2fab56e354f4fac76ad775d3f39c5/users/88c0001a6d384e39b132bb0319eeef9b/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:53:33.795290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d53994-e2e6-4cf0-9920-3fe0c0bfc63b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.795664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d53994-e2e6-4cf0-9920-3fe0c0bfc63b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.796036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d53994-e2e6-4cf0-9920-3fe0c0bfc63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.827219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-62d53994-e2e6-4cf0-9920-3fe0c0bfc63b None None] MFA Rules not processed for user `88c0001a6d384e39b132bb0319eeef9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:33.828443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-62d53994-e2e6-4cf0-9920-3fe0c0bfc63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:33.852950 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:33.896980 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1221/2463] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:33.901871 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:33.967880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ee091-559f-4961-b0ee-e65fd95d135b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:33.972675 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:33.973383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:33.973672 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:33.973953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:33.974659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:33.976581 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:34.063169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1243/2464] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:33 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:34.067199 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d53994-e2e6-4cf0-9920-3fe0c0bfc63b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:34.079002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d53994-e2e6-4cf0-9920-3fe0c0bfc63b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:34.135786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:34.136467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.136785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.137071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.137769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:34.139798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:34.237917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1222/2465] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:34 2022] POST /identity/v3/users => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:34.241908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:34.255717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dbe52-402c-4418-bdae-7d9bd8dd7a72 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:34.261826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:34.262596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.262889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.263212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.263948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:34.265619 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:34.276991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1244/2466] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:34.281157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:34.296656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf28120-2c0b-4abb-b8e4-aa1340accd97 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:34.296893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:34.297731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.298019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.298303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] PATH_INFO: `/v3/projects/46f0ebcdc76641d6aa6569da96221405/users/b0ad9ae2a8af442cb1dcbe48c4453c9c/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.313655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f0ebcdc76641d6aa6569da96221405, user_id=b0ad9ae2a8af442cb1dcbe48c4453c9c, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:34.316368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:34.386937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1223/2467] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:34 2022] PUT /identity/v3/projects/46f0ebcdc76641d6aa6569da96221405/users/b0ad9ae2a8af442cb1dcbe48c4453c9c/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:53:34.390745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:34.400848 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056d965-a427-461b-ba20-e74a60997819 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:34.468102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3433c46-bf91-4bbe-b4d4-46e9c08fb399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:34.468803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433c46-bf91-4bbe-b4d4-46e9c08fb399 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.469117 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433c46-bf91-4bbe-b4d4-46e9c08fb399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.469466 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433c46-bf91-4bbe-b4d4-46e9c08fb399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.470198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3433c46-bf91-4bbe-b4d4-46e9c08fb399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:34.471872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3433c46-bf91-4bbe-b4d4-46e9c08fb399 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:34.487178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1245/2468] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:34.488624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:34.510227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5e48-be38-490a-a1a4-39254cc008fd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:34.516705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:34.517612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.517901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.518181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] PATH_INFO: `/v3/projects/46f0ebcdc76641d6aa6569da96221405/users/b0ad9ae2a8af442cb1dcbe48c4453c9c/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.520351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f0ebcdc76641d6aa6569da96221405, user_id=b0ad9ae2a8af442cb1dcbe48c4453c9c, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:34.523121 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:34.534910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1224/2469] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:34 2022] PUT /identity/v3/projects/46f0ebcdc76641d6aa6569da96221405/users/b0ad9ae2a8af442cb1dcbe48c4453c9c/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:53:34.546936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804612c7-52ad-4b1f-a51a-b23a9813be2e None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.547068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804612c7-52ad-4b1f-a51a-b23a9813be2e None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.547245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804612c7-52ad-4b1f-a51a-b23a9813be2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.578066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-804612c7-52ad-4b1f-a51a-b23a9813be2e None None] MFA Rules not processed for user `b0ad9ae2a8af442cb1dcbe48c4453c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:34.579590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-804612c7-52ad-4b1f-a51a-b23a9813be2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:34.632586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:34.654490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1246/2470] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:34.665543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:34.667269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:34.731540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e231-a508-4c8b-b023-d67cc90d4914 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:34.736286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd9ffd6-bdc6-40df-9f29-1ff759962e6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:34.737493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9ffd6-bdc6-40df-9f29-1ff759962e6f service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.737990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9ffd6-bdc6-40df-9f29-1ff759962e6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.738565 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9ffd6-bdc6-40df-9f29-1ff759962e6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.754484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd9ffd6-bdc6-40df-9f29-1ff759962e6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:34.755971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd9ffd6-bdc6-40df-9f29-1ff759962e6f service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:34.766789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1225/2471] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:34.893512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804612c7-52ad-4b1f-a51a-b23a9813be2e None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:34.970205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804612c7-52ad-4b1f-a51a-b23a9813be2e None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:34.976314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:34.977236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:34.977523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:34.977838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] PATH_INFO: `/v3/users/88c0001a6d384e39b132bb0319eeef9b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:34.979059 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c0001a6d384e39b132bb0319eeef9b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:34.981178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:35.192240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c0001a6d384e39b132bb0319eeef9b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:35.208255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c0001a6d384e39b132bb0319eeef9b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:35.212963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c0001a6d384e39b132bb0319eeef9b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:35.219298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c0001a6d384e39b132bb0319eeef9b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:35.220312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c0001a6d384e39b132bb0319eeef9b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:35.302470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1247/2472] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:34 2022] DELETE /identity/v3/users/88c0001a6d384e39b132bb0319eeef9b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:35.307882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:35.308198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:35.308444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:35.327202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:35.328456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:35.343638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:35.402412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1226/2473] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:35.412893 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:35.416015 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:35.479322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db490d-1697-4067-a21a-a00324fb14cd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:35.483058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6da57d-bd5a-4a77-af22-3d843581f0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:35.483711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6da57d-bd5a-4a77-af22-3d843581f0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:35.483995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6da57d-bd5a-4a77-af22-3d843581f0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:35.484274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6da57d-bd5a-4a77-af22-3d843581f0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:35.494954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6da57d-bd5a-4a77-af22-3d843581f0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:35.496408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6da57d-bd5a-4a77-af22-3d843581f0c8 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:35.506838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1248/2474] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:35.937664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:35.939296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:35.954900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:35.954900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:35.954900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:35.955276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b7cd-0b9b-45f7-8bc2-e1b3a0130830 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:35.960387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:35.961639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:35.962478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:35.962891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] PATH_INFO: `/v3/projects/45a2fab56e354f4fac76ad775d3f39c5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:35.964327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a2fab56e354f4fac76ad775d3f39c5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:35.968944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:35.975034 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:35.976098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:35.978410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:36.036256 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1249/2475] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:53:36.049412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:36.050832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:36.066704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a2fab56e354f4fac76ad775d3f39c5', 'request_id': 'req-8277d8d5-465f-46d0-a406-2160ff4d48b7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.073312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a2fab56e354f4fac76ad775d3f39c5', 'request_id': 'req-8277d8d5-465f-46d0-a406-2160ff4d48b7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.126131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e65ce-0525-404c-9df2-3d12cb4a26b5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:36.128800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a2fab56e354f4fac76ad775d3f39c5', 'request_id': 'req-8277d8d5-465f-46d0-a406-2160ff4d48b7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.142781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] The token cache is being invalidate because project 45a2fab56e354f4fac76ad775d3f39c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:36.143427 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.144518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.147218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1227/2476] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:35 2022] DELETE /identity/v3/projects/45a2fab56e354f4fac76ad775d3f39c5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:36.151537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:36.152918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:36.202351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:36.202906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:36.203622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:36.203912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:36.206072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:36.230722 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277d8d5-465f-46d0-a406-2160ff4d48b7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:36.236148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:36.237231 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:36.237528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:36.237813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] PATH_INFO: `/v3/users/b0ad9ae2a8af442cb1dcbe48c4453c9c` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:36.239836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ad9ae2a8af442cb1dcbe48c4453c9c)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:36.241675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:36.282832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:36.285601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:36.287202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:36.296580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1250/2477] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:53:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:53:36.437540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:36.439678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ad9ae2a8af442cb1dcbe48c4453c9c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.448354 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ad9ae2a8af442cb1dcbe48c4453c9c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.512581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ad9ae2a8af442cb1dcbe48c4453c9c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.513322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ad9ae2a8af442cb1dcbe48c4453c9c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.514395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-6a2fa980-7ba6-4fc5-ad6d-b55e243f37e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ad9ae2a8af442cb1dcbe48c4453c9c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:36.547123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1228/2478] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:36 2022] DELETE /identity/v3/users/b0ad9ae2a8af442cb1dcbe48c4453c9c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:37.067584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:37.108281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06bfe3-f890-410c-9b48-71facd00f61c service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:37.183081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:37.184035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.184430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.184753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] PATH_INFO: `/v3/projects/46f0ebcdc76641d6aa6569da96221405` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.185952 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f0ebcdc76641d6aa6569da96221405)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:37.188018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:37.311966 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f0ebcdc76641d6aa6569da96221405', 'request_id': 'req-f2608f73-8d3a-4259-956c-3c89c8d95051'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:37.319570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f0ebcdc76641d6aa6569da96221405', 'request_id': 'req-f2608f73-8d3a-4259-956c-3c89c8d95051'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:37.325854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f0ebcdc76641d6aa6569da96221405', 'request_id': 'req-f2608f73-8d3a-4259-956c-3c89c8d95051'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:37.387498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] The token cache is being invalidate because project 46f0ebcdc76641d6aa6569da96221405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:37.387934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:37.388486 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f2608f73-8d3a-4259-956c-3c89c8d95051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:37.390512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1251/2479] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:37 2022] DELETE /identity/v3/projects/46f0ebcdc76641d6aa6569da96221405 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:37.395532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c6c19-2847-491b-9919-b5a8c5f1f23e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.396081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c6c19-2847-491b-9919-b5a8c5f1f23e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.397185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c6c19-2847-491b-9919-b5a8c5f1f23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.433799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c23c6c19-2847-491b-9919-b5a8c5f1f23e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:37.435226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c23c6c19-2847-491b-9919-b5a8c5f1f23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:37.517721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1229/2480] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:37.526976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83eb85-7b1b-4c39-b5fe-c2ccfab25c5f None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.527343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83eb85-7b1b-4c39-b5fe-c2ccfab25c5f None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.527631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83eb85-7b1b-4c39-b5fe-c2ccfab25c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.547250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-aa83eb85-7b1b-4c39-b5fe-c2ccfab25c5f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:37.548551 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aa83eb85-7b1b-4c39-b5fe-c2ccfab25c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:37.557563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1252/2481] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:37.561933 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c6c19-2847-491b-9919-b5a8c5f1f23e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:37.572981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c6c19-2847-491b-9919-b5a8c5f1f23e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:37.577576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:37.578266 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.578554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.578838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.579566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:53:37.579883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:37.581570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:37.593899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1230/2482] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:53:37.598254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83eb85-7b1b-4c39-b5fe-c2ccfab25c5f None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:37.608301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83eb85-7b1b-4c39-b5fe-c2ccfab25c5f None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:37.613179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:37.613840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.614129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.614585 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.615287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:37.617379 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:37.692212 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1253/2483] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:37 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:37.696049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:37.707609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5132f81-6a12-48a4-9e87-077d3469bc8f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:37.765309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:37.766011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.766316 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.766592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.767329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:37.769325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:37.884878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1231/2484] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:37 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:37.891055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:37.922779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037a4c1-c3f1-401f-aac2-0d5eb109b740 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:37.938367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:37.939746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.940259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.940789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.941983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:37.943982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:37.961185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1254/2485] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:37.962836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:37.975097 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f699a5-7096-4d45-81c1-3771d0e1dbcd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:37.992337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761406b-592e-42ef-a671-d91cad25ee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:37.993071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761406b-592e-42ef-a671-d91cad25ee62 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:37.993932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761406b-592e-42ef-a671-d91cad25ee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:37.994211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761406b-592e-42ef-a671-d91cad25ee62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:37.998323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761406b-592e-42ef-a671-d91cad25ee62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:37.998323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761406b-592e-42ef-a671-d91cad25ee62 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:38.009553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1232/2486] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:38.013667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:38.026474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea79d3-e129-471c-bd79-2e164fb6a85a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:38.036971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:38.038188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:38.038646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:38.039104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] PATH_INFO: `/v3/projects/9ee239f0b01b4168812a8e9a0db61914/users/edb9d8b12afb48d8a977d60eb06a5203/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:38.058633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee239f0b01b4168812a8e9a0db61914, user_id=edb9d8b12afb48d8a977d60eb06a5203, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:38.061494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:38.135376 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1255/2487] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:38 2022] PUT /identity/v3/projects/9ee239f0b01b4168812a8e9a0db61914/users/edb9d8b12afb48d8a977d60eb06a5203/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:38.145798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10d56-74ce-4c68-81f4-c805b428001d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:38.146101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10d56-74ce-4c68-81f4-c805b428001d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:38.146388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10d56-74ce-4c68-81f4-c805b428001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:38.177400 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-b5b10d56-74ce-4c68-81f4-c805b428001d None None] MFA Rules not processed for user `edb9d8b12afb48d8a977d60eb06a5203`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:38.180113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-b5b10d56-74ce-4c68-81f4-c805b428001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:38.196024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:38.259003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1233/2488] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:38.269573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:38.272899 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:38.324050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:38.360139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f77fb-00cd-4c4c-a5fa-dbe60ca41dc6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:38.363888 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a5a91-0c38-43ee-a2ac-548f50e490ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:38.364682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a5a91-0c38-43ee-a2ac-548f50e490ab service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:38.364973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a5a91-0c38-43ee-a2ac-548f50e490ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:38.365252 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a5a91-0c38-43ee-a2ac-548f50e490ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:38.374071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a5a91-0c38-43ee-a2ac-548f50e490ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:38.376076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a5a91-0c38-43ee-a2ac-548f50e490ab service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:38.387131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1256/2489] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:42.333433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10d56-74ce-4c68-81f4-c805b428001d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:42.397006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10d56-74ce-4c68-81f4-c805b428001d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:42.402100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:42.402974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:42.403262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:42.403573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] PATH_INFO: `/v3/users/edb9d8b12afb48d8a977d60eb06a5203` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:42.404802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb9d8b12afb48d8a977d60eb06a5203)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:42.406774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:42.611093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb9d8b12afb48d8a977d60eb06a5203'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:42.618633 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb9d8b12afb48d8a977d60eb06a5203'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:42.671501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb9d8b12afb48d8a977d60eb06a5203'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:42.672150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb9d8b12afb48d8a977d60eb06a5203'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:42.672793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb9d8b12afb48d8a977d60eb06a5203'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:42.692253 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1234/2490] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:42 2022] DELETE /identity/v3/users/edb9d8b12afb48d8a977d60eb06a5203 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:42.697472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:42.697963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:42.698357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:42.718120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:42.719354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:42.800413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1257/2491] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:42.810537 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:42.811974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:42.857129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:42.872616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2c196-39b5-46c4-895a-e3f871b65405 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:42.876515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4faec5-f5d6-4ca8-9ec3-20cf792e3a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:42.877185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4faec5-f5d6-4ca8-9ec3-20cf792e3a05 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:42.877472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4faec5-f5d6-4ca8-9ec3-20cf792e3a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:42.877758 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4faec5-f5d6-4ca8-9ec3-20cf792e3a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:42.887591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4faec5-f5d6-4ca8-9ec3-20cf792e3a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:42.891443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4faec5-f5d6-4ca8-9ec3-20cf792e3a05 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:42.907182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1235/2492] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:43.337580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:43.339238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:43.364091 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07aaebe-3f24-45bf-86b1-c000a1a6b3cd None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:43.377361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:43.378267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:43.381831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:43.381831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] PATH_INFO: `/v3/projects/9ee239f0b01b4168812a8e9a0db61914` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:43.381831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee239f0b01b4168812a8e9a0db61914)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:43.382179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:43.562264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee239f0b01b4168812a8e9a0db61914', 'request_id': 'req-6962594b-013b-4e4b-96a5-34f6626ae18c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:43.572081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee239f0b01b4168812a8e9a0db61914', 'request_id': 'req-6962594b-013b-4e4b-96a5-34f6626ae18c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:43.579898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee239f0b01b4168812a8e9a0db61914', 'request_id': 'req-6962594b-013b-4e4b-96a5-34f6626ae18c'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:44.081573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] The token cache is being invalidate because project 9ee239f0b01b4168812a8e9a0db61914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:44.081573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:44.081573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6962594b-013b-4e4b-96a5-34f6626ae18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:44.084084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1258/2493] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:43 2022] DELETE /identity/v3/projects/9ee239f0b01b4168812a8e9a0db61914 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:44.089904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582d1aa-7cfd-488d-8710-42791b09ce40 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.090384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582d1aa-7cfd-488d-8710-42791b09ce40 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.090495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582d1aa-7cfd-488d-8710-42791b09ce40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.111402 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c582d1aa-7cfd-488d-8710-42791b09ce40 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:44.116019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c582d1aa-7cfd-488d-8710-42791b09ce40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:44.187811 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1236/2494] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:44.195928 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950d985-b3df-4134-954b-8fc9118c08e2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.196317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950d985-b3df-4134-954b-8fc9118c08e2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.196664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950d985-b3df-4134-954b-8fc9118c08e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.217584 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0950d985-b3df-4134-954b-8fc9118c08e2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:44.219667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0950d985-b3df-4134-954b-8fc9118c08e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:44.228628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1259/2495] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:44.233276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582d1aa-7cfd-488d-8710-42791b09ce40 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:44.251397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582d1aa-7cfd-488d-8710-42791b09ce40 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:44.256947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:44.257645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.257935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.258194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.259048 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:53:44.259414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:44.261484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:44.277543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1237/2496] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:53:44.282483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950d985-b3df-4134-954b-8fc9118c08e2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:44.296154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950d985-b3df-4134-954b-8fc9118c08e2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:44.301397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:44.302083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.302375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.302666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.303514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:44.306057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:44.391179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1260/2497] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:44 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:44.392789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:44.406101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb0d5-8385-4321-aa13-3ae90b4f97d9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:44.479114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:44.479897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.480323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.480625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.481683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:44.484718 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:44.595748 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1238/2498] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:44 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:44.599755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:44.614586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e221-5839-461d-bfe4-36db2bdc27f1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:44.623348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:44.624185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.624962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.625312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.626173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:44.628127 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:44.650888 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1261/2499] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:44.654983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:44.667844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b4da2-9ac3-4371-885e-fe2f124de1e2 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:44.675668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47440ff-6b7d-4cac-b573-235207aa5dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:44.676543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47440ff-6b7d-4cac-b573-235207aa5dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.677508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47440ff-6b7d-4cac-b573-235207aa5dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.677508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47440ff-6b7d-4cac-b573-235207aa5dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.678315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47440ff-6b7d-4cac-b573-235207aa5dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:44.680038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47440ff-6b7d-4cac-b573-235207aa5dc1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:44.697786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1239/2500] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:44.699281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:44.714152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd932eb-fe8f-4119-a709-33189e15b27f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:44.720249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:44.722014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.722309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.722618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] PATH_INFO: `/v3/projects/99b9622b629248cbbf5159242d9903fd/users/c44af7bfe0a64de48c4b9026ba5d4d80/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.726929 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:44.741359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b9622b629248cbbf5159242d9903fd, user_id=c44af7bfe0a64de48c4b9026ba5d4d80, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:44.743386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:44.826555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1262/2501] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:44 2022] PUT /identity/v3/projects/99b9622b629248cbbf5159242d9903fd/users/c44af7bfe0a64de48c4b9026ba5d4d80/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:44.838613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd54d6-1636-4b98-8ec7-bba342e9f09e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:44.838911 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd54d6-1636-4b98-8ec7-bba342e9f09e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:44.839198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd54d6-1636-4b98-8ec7-bba342e9f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:44.868523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-54fd54d6-1636-4b98-8ec7-bba342e9f09e None None] MFA Rules not processed for user `c44af7bfe0a64de48c4b9026ba5d4d80`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:44.880553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-54fd54d6-1636-4b98-8ec7-bba342e9f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:44.918895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:44.952024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1240/2502] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:44.965229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:44.965229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:45.038724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cdfa8-66ee-4f34-ac5e-8d5ae9bc4032 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:45.045313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:45.047931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:45.048304 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:45.048681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:45.064411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:45.066241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:45.081905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1263/2503] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:54.554620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc48984-349e-43c5-95e7-86ecef6e41d0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:54.555089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc48984-349e-43c5-95e7-86ecef6e41d0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:54.555185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc48984-349e-43c5-95e7-86ecef6e41d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:54.575600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7fc48984-349e-43c5-95e7-86ecef6e41d0 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:54.576851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7fc48984-349e-43c5-95e7-86ecef6e41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:54.635041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1241/2504] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 15 17:53:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:54.653512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:54.653512 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:54.721572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7846f-31eb-40e3-b2a2-6301f94db35a service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:54.727053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1cf50-5c6e-4c62-8617-02d8f5a92492 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:54.728621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cf50-5c6e-4c62-8617-02d8f5a92492 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:54.729167 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cf50-5c6e-4c62-8617-02d8f5a92492 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:54.729458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cf50-5c6e-4c62-8617-02d8f5a92492 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:54.737823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1cf50-5c6e-4c62-8617-02d8f5a92492 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:54.739462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1cf50-5c6e-4c62-8617-02d8f5a92492 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:54.747753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1264/2505] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:53:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:53:57.261988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc48984-349e-43c5-95e7-86ecef6e41d0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:57.325259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc48984-349e-43c5-95e7-86ecef6e41d0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:57.329839 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:57.330725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:57.331011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:57.331311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] PATH_INFO: `/v3/users/c44af7bfe0a64de48c4b9026ba5d4d80` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:57.332559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44af7bfe0a64de48c4b9026ba5d4d80)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:57.334618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:57.485547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44af7bfe0a64de48c4b9026ba5d4d80'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:57.494621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44af7bfe0a64de48c4b9026ba5d4d80'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:57.550807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44af7bfe0a64de48c4b9026ba5d4d80'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:57.551501 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44af7bfe0a64de48c4b9026ba5d4d80'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:57.552159 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44af7bfe0a64de48c4b9026ba5d4d80'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:57.567017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:57.571250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1242/2506] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:53:57 2022] DELETE /identity/v3/users/c44af7bfe0a64de48c4b9026ba5d4d80 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:57.575753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:57.576238 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:57.576348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:57.593654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:57.594903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:57.654890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1265/2507] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:57.665275 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:57.666755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:57.724294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64e5f3-7f9c-4318-b30a-d86448c51328 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:57.728472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207d4c-e181-40a5-9ce8-da99f3d87816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:57.729156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207d4c-e181-40a5-9ce8-da99f3d87816 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:57.729445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207d4c-e181-40a5-9ce8-da99f3d87816 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:57.729727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207d4c-e181-40a5-9ce8-da99f3d87816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:57.739098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207d4c-e181-40a5-9ce8-da99f3d87816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:57.740452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207d4c-e181-40a5-9ce8-da99f3d87816 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:57.750137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1243/2508] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:53:58.118860 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:58.120288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:58.132536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b968399-a6bb-4cbb-b211-635ad276ebae None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:58.139029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:58.139897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:58.140187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:58.140465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] PATH_INFO: `/v3/projects/99b9622b629248cbbf5159242d9903fd` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:58.141830 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b9622b629248cbbf5159242d9903fd)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:58.146831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:58.158671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:58.274799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b9622b629248cbbf5159242d9903fd', 'request_id': 'req-205dc8bd-6fec-4525-9fee-9485d002e2cf'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:58.282570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b9622b629248cbbf5159242d9903fd', 'request_id': 'req-205dc8bd-6fec-4525-9fee-9485d002e2cf'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:58.289030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b9622b629248cbbf5159242d9903fd', 'request_id': 'req-205dc8bd-6fec-4525-9fee-9485d002e2cf'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:58.381861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] The token cache is being invalidate because project 99b9622b629248cbbf5159242d9903fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:53:58.382342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:58.383810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-205dc8bd-6fec-4525-9fee-9485d002e2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:53:58.385567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1266/2509] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:53:58 2022] DELETE /identity/v3/projects/99b9622b629248cbbf5159242d9903fd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:58.392113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec36cb-5b32-4251-bcd4-537cced356e9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:58.393226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec36cb-5b32-4251-bcd4-537cced356e9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:58.393675 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec36cb-5b32-4251-bcd4-537cced356e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:58.430655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-09ec36cb-5b32-4251-bcd4-537cced356e9 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:58.430655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-09ec36cb-5b32-4251-bcd4-537cced356e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:58.503542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1244/2510] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:58.512492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab557f8-d7e8-4ab9-8f5c-eb07e5c3d3e5 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:58.512815 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab557f8-d7e8-4ab9-8f5c-eb07e5c3d3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:58.516410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab557f8-d7e8-4ab9-8f5c-eb07e5c3d3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:58.541864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-6ab557f8-d7e8-4ab9-8f5c-eb07e5c3d3e5 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:58.550871 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6ab557f8-d7e8-4ab9-8f5c-eb07e5c3d3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:58.552071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1267/2511] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:58.556921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec36cb-5b32-4251-bcd4-537cced356e9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:58.567322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec36cb-5b32-4251-bcd4-537cced356e9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:58.575495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:58.576185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:58.576472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:58.576795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:58.577540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:53:58.585254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:58.585254 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:58.593851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1245/2512] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:53:58.598477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab557f8-d7e8-4ab9-8f5c-eb07e5c3d3e5 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:58.608447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab557f8-d7e8-4ab9-8f5c-eb07e5c3d3e5 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:58.654058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:58.654058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:58.654058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:58.654058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:58.655107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:58.655107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:58.782251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1268/2513] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:53:58 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:53:58.786277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:58.796390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe530c2-07a7-49ab-9fb6-58d4c3d6898e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:58.854398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:58.855083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:58.855403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:58.856683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:58.858111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:58.864552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:58.968284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1246/2514] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:53:58 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:53:58.972491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:58.985459 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a89a3b-1fdf-46d1-afbb-15576b97585b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:58.989991 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:58.990654 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:58.990937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:58.991214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:58.991994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:58.994360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:59.005743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1269/2515] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:59.009957 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:59.020061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11d168-1666-4368-bcb6-6212c9370ff5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:59.025797 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaac593-decd-4e6e-924d-672f9ec3271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:59.026471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac593-decd-4e6e-924d-672f9ec3271e admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:59.026761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac593-decd-4e6e-924d-672f9ec3271e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:59.027042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac593-decd-4e6e-924d-672f9ec3271e admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:59.027774 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaac593-decd-4e6e-924d-672f9ec3271e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:59.029433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaac593-decd-4e6e-924d-672f9ec3271e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:59.044480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1247/2516] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:53:59.048611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:59.059634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238990d-53af-4434-aa80-9c1fb0d7c8f6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:59.064203 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:59.065135 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:59.065417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:59.065695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] PATH_INFO: `/v3/projects/08c91c6b348e4840939870bf231d5a91/users/98b93d5a1e5b4c46b505cdacf707ee1a/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:59.081659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c91c6b348e4840939870bf231d5a91, user_id=98b93d5a1e5b4c46b505cdacf707ee1a, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:59.084489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:59.146221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1270/2517] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:53:59 2022] PUT /identity/v3/projects/08c91c6b348e4840939870bf231d5a91/users/98b93d5a1e5b4c46b505cdacf707ee1a/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:53:59.151373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fec1f-92df-4005-a1d6-272d1229244c None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:59.151668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fec1f-92df-4005-a1d6-272d1229244c None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:59.153971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fec1f-92df-4005-a1d6-272d1229244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:59.173613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-744fec1f-92df-4005-a1d6-272d1229244c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:53:59.174830 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-744fec1f-92df-4005-a1d6-272d1229244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:59.179093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:53:59.239028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1248/2518] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:53:59.251052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:53:59.256189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:53:59.310180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cac96-32bf-4b4f-aaf8-48a4f7de8d08 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:53:59.313757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:53:59.314399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:53:59.314680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:53:59.314957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:53:59.323889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:53:59.325350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:53:59.334915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1271/2519] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:06.026676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c9cf-1fc0-4344-be29-7ebb822af26d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:06.027733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c9cf-1fc0-4344-be29-7ebb822af26d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:06.028246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08c9cf-1fc0-4344-be29-7ebb822af26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:06.058174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3a08c9cf-1fc0-4344-be29-7ebb822af26d None None] MFA Rules not processed for user `98b93d5a1e5b4c46b505cdacf707ee1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:06.059773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3a08c9cf-1fc0-4344-be29-7ebb822af26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:06.134535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1249/2520] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:06.139838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:06.154402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4bc5c-87b0-40a4-8059-632aaf98e57d service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:06.159455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:06.160800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:06.161282 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:06.161634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:06.162599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:06.165698 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:06.276836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1272/2521] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:06 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:06.278049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c9cf-1fc0-4344-be29-7ebb822af26d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:06.289945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08c9cf-1fc0-4344-be29-7ebb822af26d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:06.351493 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:06.353621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:06.353621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:06.353621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:06.354182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:06.356228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:06.463251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1250/2522] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:06 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:06.467254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:06.483579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce7454-0734-454d-a8e3-dd4494fb7e16 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:06.489273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:06.489988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:06.490276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:06.490557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:06.491393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:06.493681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:06.506600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1273/2523] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:06.510746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:06.521204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c0548-c73a-4559-a971-b015eeb48433 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:06.527858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:06.528792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:06.529080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:06.529400 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] PATH_INFO: `/v3/projects/69ae8ad9edfa4ec690d8635cabc281cd/users/f5b39c42e897405fa510fb8fd18b4217/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:06.547247 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ae8ad9edfa4ec690d8635cabc281cd, user_id=f5b39c42e897405fa510fb8fd18b4217, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:06.550335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:06.779292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1251/2524] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:06 2022] PUT /identity/v3/projects/69ae8ad9edfa4ec690d8635cabc281cd/users/f5b39c42e897405fa510fb8fd18b4217/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:06.780420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:06.793184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded966e-d3b5-43c4-ba33-9fc2a8b1d649 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:06.799354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:06.955134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b7636-b8d0-4460-8bf5-5e3e3adf9495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:06.957222 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b7636-b8d0-4460-8bf5-5e3e3adf9495 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:06.957518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b7636-b8d0-4460-8bf5-5e3e3adf9495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:06.957800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b7636-b8d0-4460-8bf5-5e3e3adf9495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:06.958994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b7636-b8d0-4460-8bf5-5e3e3adf9495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:06.961571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b7636-b8d0-4460-8bf5-5e3e3adf9495 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:06.974861 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1274/2525] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:06.979486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:06.986434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:06.990996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3dc1f-d012-4cb9-8cde-2f22714a81a6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:07.000022 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:07.001000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:07.001293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:07.001582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] PATH_INFO: `/v3/projects/69ae8ad9edfa4ec690d8635cabc281cd/users/f5b39c42e897405fa510fb8fd18b4217/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:07.005796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ae8ad9edfa4ec690d8635cabc281cd, user_id=f5b39c42e897405fa510fb8fd18b4217, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:07.017478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:07.057601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1252/2526] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:06 2022] PUT /identity/v3/projects/69ae8ad9edfa4ec690d8635cabc281cd/users/f5b39c42e897405fa510fb8fd18b4217/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:13.060634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:13.065269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:13.065269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:13.103676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] MFA Rules not processed for user `f5b39c42e897405fa510fb8fd18b4217`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:13.104948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:13.215312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1275/2527] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:13.235936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:13.237904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:13.303207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae278b-5d56-4eae-b3eb-9cc3657690e8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:13.307107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f73173-eb08-4762-96ca-e038d5c87d5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:13.307761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f73173-eb08-4762-96ca-e038d5c87d5f service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:13.308052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f73173-eb08-4762-96ca-e038d5c87d5f service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:13.308331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f73173-eb08-4762-96ca-e038d5c87d5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:13.380006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f73173-eb08-4762-96ca-e038d5c87d5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:13.381381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f73173-eb08-4762-96ca-e038d5c87d5f service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:13.393064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1253/2528] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:13.669534 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:13.674556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:13.722484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:13.722780 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:13.723063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:13.745954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:13.747479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:13.748671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1c98d-470c-4665-b543-c653dc7d3771 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:13.753948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:13.754341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:13.754624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:13.754903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:13.757004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:13.803646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:13.820698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1254/2529] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:54:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:13.828273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:13.829758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:13.833666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:13.835363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:13.842757 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1276/2530] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:13.896479 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:13.916708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c9773-46b8-497e-8808-1c2ad88e43a9 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:13.922197 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:13.922867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:13.923151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:13.923428 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:13.932472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:13.934114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:13.955848 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1255/2531] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:54:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 17:54:20.419536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:20.421063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:20.446503 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c72da2-8a1b-449c-ac81-a360ab40c17b service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:20.462682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:20.464123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:20.464692 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:20.465205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] PATH_INFO: `/v3/users/5013f4ba5cef44f7bdab6531ec0b0836` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:20.466900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5013f4ba5cef44f7bdab6531ec0b0836)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:20.469456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:20.755790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5013f4ba5cef44f7bdab6531ec0b0836'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:20.781572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5013f4ba5cef44f7bdab6531ec0b0836'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:20.782582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5013f4ba5cef44f7bdab6531ec0b0836'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:20.798010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5013f4ba5cef44f7bdab6531ec0b0836'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:20.798776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-aa4e1d4e-fceb-43af-bb79-226943693ebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5013f4ba5cef44f7bdab6531ec0b0836'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:20.910577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1277/2532] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:20 2022] DELETE /identity/v3/users/5013f4ba5cef44f7bdab6531ec0b0836 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:21.356626 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:21.377153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ca8a6-0550-44fd-a926-08dd0b4e8a04 service swift] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:21.558103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:21.561339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:21.561639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:21.561929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] PATH_INFO: `/v3/projects/fbcdf72827044c04842b74194eaf13c3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:21.568010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbcdf72827044c04842b74194eaf13c3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:21.570108 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:21.823680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbcdf72827044c04842b74194eaf13c3', 'request_id': 'req-a5c40bc8-e93c-416d-9e0d-6738958c0dae'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:21.837978 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbcdf72827044c04842b74194eaf13c3', 'request_id': 'req-a5c40bc8-e93c-416d-9e0d-6738958c0dae'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:21.914483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbcdf72827044c04842b74194eaf13c3', 'request_id': 'req-a5c40bc8-e93c-416d-9e0d-6738958c0dae'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:21.930337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] The token cache is being invalidate because project fbcdf72827044c04842b74194eaf13c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:21.930623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:21.931284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a5c40bc8-e93c-416d-9e0d-6738958c0dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:21.933424 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1256/2533] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:21 2022] DELETE /identity/v3/projects/fbcdf72827044c04842b74194eaf13c3 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:21.938234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7e09-f596-42b0-a0b5-ac481dea1687 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:21.938744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7e09-f596-42b0-a0b5-ac481dea1687 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:21.938840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7e09-f596-42b0-a0b5-ac481dea1687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:21.969253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-74de7e09-f596-42b0-a0b5-ac481dea1687 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:21.970959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-74de7e09-f596-42b0-a0b5-ac481dea1687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:22.083336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1278/2534] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:22.084583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170fa5c-3058-44d2-88b2-db268efe0af5 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.084923 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170fa5c-3058-44d2-88b2-db268efe0af5 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.085207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170fa5c-3058-44d2-88b2-db268efe0af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.107372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-d170fa5c-3058-44d2-88b2-db268efe0af5 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:22.108660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d170fa5c-3058-44d2-88b2-db268efe0af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:22.121872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1257/2535] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:22.126026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7e09-f596-42b0-a0b5-ac481dea1687 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:22.141032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7e09-f596-42b0-a0b5-ac481dea1687 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:22.146401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:22.147494 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.149410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.153517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.154559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:22.154944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:22.156756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:22.171740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1279/2536] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:54:22.175692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170fa5c-3058-44d2-88b2-db268efe0af5 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:22.192303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170fa5c-3058-44d2-88b2-db268efe0af5 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:22.200296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:22.201368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.201824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.202352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.203460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:22.205624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:22.311654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1258/2537] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:22 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:22.315462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:22.330548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83d11-a80a-40ba-abfc-26d4b0f08c12 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:22.397600 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:22.429971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:22.430964 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.431408 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.431839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.432785 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:22.437742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:22.509726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1280/2538] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:22 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:22.513832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:22.534949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9daeb-bf42-4937-bb85-f791b9e6d75f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:22.543471 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:22.545094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.545590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.546144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.547116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:22.549094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:22.588187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1259/2539] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:22.596901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:22.613430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4451cec-452f-4695-adac-54709c6a5e98 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:22.625436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b1515-8fa2-4356-a7b4-0e25a8cdaf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:22.628748 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b1515-8fa2-4356-a7b4-0e25a8cdaf54 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.629346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b1515-8fa2-4356-a7b4-0e25a8cdaf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.629641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b1515-8fa2-4356-a7b4-0e25a8cdaf54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.630399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b1515-8fa2-4356-a7b4-0e25a8cdaf54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:22.633087 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b1515-8fa2-4356-a7b4-0e25a8cdaf54 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:22.655175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1281/2540] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:22.659770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:22.675700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45361b-8e69-40dc-964b-0a0ac41df4d7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:22.682229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:22.684661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.685027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.685381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] PATH_INFO: `/v3/projects/626b48791ae34848bf321284976c2d72/users/b7d18217542243a29f3195c1c9fdac51/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.704150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626b48791ae34848bf321284976c2d72, user_id=b7d18217542243a29f3195c1c9fdac51, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:22.707417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:22.788464 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1260/2541] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:22 2022] PUT /identity/v3/projects/626b48791ae34848bf321284976c2d72/users/b7d18217542243a29f3195c1c9fdac51/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:22.800180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ac3b-cb91-4f5a-8d8a-9c53983219dc None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:22.800498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ac3b-cb91-4f5a-8d8a-9c53983219dc None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:22.801017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ac3b-cb91-4f5a-8d8a-9c53983219dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:22.838301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3391ac3b-cb91-4f5a-8d8a-9c53983219dc None None] MFA Rules not processed for user `b7d18217542243a29f3195c1c9fdac51`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:22.839917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3391ac3b-cb91-4f5a-8d8a-9c53983219dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:22.941805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:22.944230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1282/2542] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:22.954262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:23.065754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cb367-af9e-4325-a07b-a61293a2ac84 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:23.072436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:23.073263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:23.073627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:23.074006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:23.074792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:23.077222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:23.199692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1261/2543] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:22 2022] POST /identity/v3/projects => generated 365 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:23.203988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ac3b-cb91-4f5a-8d8a-9c53983219dc None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:23.233982 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ac3b-cb91-4f5a-8d8a-9c53983219dc None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:23.316463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:23.316463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:23.316463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:23.316463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:23.322756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:23.322756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:23.826150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1283/2544] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:23 2022] POST /identity/v3/users => generated 289 bytes in 623 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:23.830403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:23.840930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e6dfc-a327-4576-b8bc-87645407e84c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:23.845575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:23.846228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:23.846518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:23.846793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:23.847549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:23.849246 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:23.860441 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1262/2545] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:23.864840 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:23.883610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685a2ed-7708-49dd-94ae-dc25f06269cb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:23.888325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:23.889273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:23.889559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:23.889839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] PATH_INFO: `/v3/projects/c7de33cd22654b7cb0d7fd6f05f5aec2/users/af2f4fe8e3804710839ca8b9c5be9795/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:23.910097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7de33cd22654b7cb0d7fd6f05f5aec2, user_id=af2f4fe8e3804710839ca8b9c5be9795, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:23.912947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:24.032969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1284/2546] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:23 2022] PUT /identity/v3/projects/c7de33cd22654b7cb0d7fd6f05f5aec2/users/af2f4fe8e3804710839ca8b9c5be9795/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:24.035437 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:24.048007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb84f-1faf-483c-926a-022c93148de6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:24.123430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b5ed6-0add-40d1-bf46-c108c136b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:24.124362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5ed6-0add-40d1-bf46-c108c136b3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:24.124679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5ed6-0add-40d1-bf46-c108c136b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:24.125010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5ed6-0add-40d1-bf46-c108c136b3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:24.125744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b5ed6-0add-40d1-bf46-c108c136b3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:24.127755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b5ed6-0add-40d1-bf46-c108c136b3ee admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:24.147137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1263/2547] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:24.153808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:24.163384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790486-93df-4f73-8ca4-0e433f2f2e82 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:24.169421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:24.172744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:24.173033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:24.173310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] PATH_INFO: `/v3/projects/c7de33cd22654b7cb0d7fd6f05f5aec2/users/af2f4fe8e3804710839ca8b9c5be9795/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:24.175695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7de33cd22654b7cb0d7fd6f05f5aec2, user_id=af2f4fe8e3804710839ca8b9c5be9795, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:24.182154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:24.195979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:24.203030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1285/2548] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:24 2022] PUT /identity/v3/projects/c7de33cd22654b7cb0d7fd6f05f5aec2/users/af2f4fe8e3804710839ca8b9c5be9795/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:24.217470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41844d-1d3d-4dc7-a3a9-685ce25feda6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:24.219160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41844d-1d3d-4dc7-a3a9-685ce25feda6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:24.219160 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41844d-1d3d-4dc7-a3a9-685ce25feda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:24.252558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ce41844d-1d3d-4dc7-a3a9-685ce25feda6 None None] MFA Rules not processed for user `af2f4fe8e3804710839ca8b9c5be9795`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:24.253794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ce41844d-1d3d-4dc7-a3a9-685ce25feda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:24.343538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1264/2549] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:24.357787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:24.359968 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:24.436669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb134d9-a595-4258-948e-77d8d5b2b9c1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:24.441278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbe908-70f1-4341-8d11-00838e15ac33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:24.442287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe908-70f1-4341-8d11-00838e15ac33 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:24.442578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe908-70f1-4341-8d11-00838e15ac33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:24.442854 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbe908-70f1-4341-8d11-00838e15ac33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:24.456223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbe908-70f1-4341-8d11-00838e15ac33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:24.471438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbe908-70f1-4341-8d11-00838e15ac33 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:24.471438 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1286/2550] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:24.631398 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41844d-1d3d-4dc7-a3a9-685ce25feda6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:24.637313 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:24.645673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41844d-1d3d-4dc7-a3a9-685ce25feda6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:24.650382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:24.663018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:24.663018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:24.663018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:24.726692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:24.728148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:24.739629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1265/2551] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:28.342407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:28.342935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:28.343058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:28.361939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] MFA Rules not processed for user `98b93d5a1e5b4c46b505cdacf707ee1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:28.363141 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:28.417994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1287/2552] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:28.530855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:28.532298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:28.602405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b2503-20c4-4c6d-aea1-46f8a675f64a service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:28.606346 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:28.606971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:28.607400 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:28.607685 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:28.609615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:28.682147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:28.683746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:28.694299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1266/2553] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:28.698148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:28.699592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:28.712322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f67ea-8f3e-4f57-9312-e25f10dac6f3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:28.715921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:28.716564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:28.716875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:28.717149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:28.718947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:28.730435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:28.731925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:28.740691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1288/2554] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:32.563007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:32.563875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:32.585038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78828da-879f-4f4c-a188-ce3a02eb60ff service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:32.590867 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:32.591742 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:32.592028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:32.592304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] PATH_INFO: `/v3/users/34e7019b46d743a1b6b0d3315e69053f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:32.593538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e7019b46d743a1b6b0d3315e69053f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:32.595563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:32.821725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:32.826396 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e7019b46d743a1b6b0d3315e69053f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:32.833386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e7019b46d743a1b6b0d3315e69053f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:32.905589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e7019b46d743a1b6b0d3315e69053f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:32.906283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e7019b46d743a1b6b0d3315e69053f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:32.906837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a40f3094-3501-473c-a2a7-69b3f129688c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e7019b46d743a1b6b0d3315e69053f'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:32.928032 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1267/2555] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:32 2022] DELETE /identity/v3/users/34e7019b46d743a1b6b0d3315e69053f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:33.198709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:33.212545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667207-01c7-4c5d-b28d-a572d416cc16 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:33.276123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:33.277032 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:33.277317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:33.277629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] PATH_INFO: `/v3/projects/b464e33662344d5a80bdfe413b56b5ec` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:33.278822 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b464e33662344d5a80bdfe413b56b5ec)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:33.280777 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:33.395501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:33.406206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b464e33662344d5a80bdfe413b56b5ec', 'request_id': 'req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:33.412898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b464e33662344d5a80bdfe413b56b5ec', 'request_id': 'req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:33.418608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b464e33662344d5a80bdfe413b56b5ec', 'request_id': 'req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:33.567694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] The token cache is being invalidate because project b464e33662344d5a80bdfe413b56b5ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:33.567694 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:33.568402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-0970eded-63f9-4c78-acb9-7a47a9e3b4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:33.570688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1289/2556] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:33 2022] DELETE /identity/v3/projects/b464e33662344d5a80bdfe413b56b5ec => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:33.575834 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd6d83-ccd2-4246-ac22-33587e03110a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:33.576297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd6d83-ccd2-4246-ac22-33587e03110a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:33.576527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd6d83-ccd2-4246-ac22-33587e03110a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:33.596820 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-00cd6d83-ccd2-4246-ac22-33587e03110a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:33.598351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-00cd6d83-ccd2-4246-ac22-33587e03110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:33.676715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1268/2557] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:33.681783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319fea8-de83-412a-96fa-09819055953a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:33.682079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319fea8-de83-412a-96fa-09819055953a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:33.682359 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319fea8-de83-412a-96fa-09819055953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:33.723679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-9319fea8-de83-412a-96fa-09819055953a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:33.724942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-9319fea8-de83-412a-96fa-09819055953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:33.733318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1290/2558] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:33.737375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd6d83-ccd2-4246-ac22-33587e03110a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:33.747829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd6d83-ccd2-4246-ac22-33587e03110a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:33.752927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:33.753583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:33.753870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:33.754829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:33.754973 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:33.755273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:33.756963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:33.769598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1269/2559] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:33.774205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319fea8-de83-412a-96fa-09819055953a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:33.784320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319fea8-de83-412a-96fa-09819055953a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:33.789146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:33.789802 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:33.790116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:33.790425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:33.791166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:33.793100 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:33.910558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1291/2560] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:33 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:33.911756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:33.922778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db9092-9803-4e98-b6fc-3bfb61bd7c2d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:33.997119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:34.001157 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:34.001617 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:34.002082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:34.003010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:34.005841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:34.112532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1270/2561] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:33 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:34.116353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:34.128049 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcdecd-5e04-4aa7-b0ac-2db4c8966795 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:34.133926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:34.134624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:34.134880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:34.135284 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:34.136019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:34.137849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:34.149701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1292/2562] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:34.153798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:34.166010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f73d7d-d9aa-4ac1-83a6-34967aaaf897 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:34.170995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cacd2-21cf-44ae-967e-5c11eb12123f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:34.171667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cacd2-21cf-44ae-967e-5c11eb12123f admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:34.171963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cacd2-21cf-44ae-967e-5c11eb12123f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:34.172243 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cacd2-21cf-44ae-967e-5c11eb12123f admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:34.173036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cacd2-21cf-44ae-967e-5c11eb12123f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:34.174684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cacd2-21cf-44ae-967e-5c11eb12123f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:34.187065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1271/2563] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:34.191464 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:34.202661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c41f-926f-4625-a0a4-fe01fbbb6bb5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:34.207564 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:34.208470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:34.208790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:34.209072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] PATH_INFO: `/v3/projects/c9d18877754d4c70a02358d0cee5b13e/users/d9a8fcc207544205bec6ffa95069ec7e/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:34.224800 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d18877754d4c70a02358d0cee5b13e, user_id=d9a8fcc207544205bec6ffa95069ec7e, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:34.228722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:34.334060 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1293/2564] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:34 2022] PUT /identity/v3/projects/c9d18877754d4c70a02358d0cee5b13e/users/d9a8fcc207544205bec6ffa95069ec7e/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:34.346138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ffd4a-55eb-472d-ae84-658bd3455f05 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:34.346442 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ffd4a-55eb-472d-ae84-658bd3455f05 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:34.346725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ffd4a-55eb-472d-ae84-658bd3455f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:34.376242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-614ffd4a-55eb-472d-ae84-658bd3455f05 None None] MFA Rules not processed for user `d9a8fcc207544205bec6ffa95069ec7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:34.377738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-614ffd4a-55eb-472d-ae84-658bd3455f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:34.397521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:34.456158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1272/2565] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:34.478636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:34.481047 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:34.556898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84dcbb-8a42-43ce-a8b3-0aded885d512 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:34.567559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:34.568239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:34.568527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:34.568825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:34.578733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:34.580233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:34.590583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1294/2566] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:35.673405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:35.673888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:35.673987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:35.695524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:35.697170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:35.798142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:35.799845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:35.801984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1273/2567] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:54:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:35.894202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:35.901552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9c290-d8ee-4ac3-93f0-15a9b99718c6 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:35.909536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618ca03-3ed1-4d02-85b6-69d180b3f44e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:35.911030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618ca03-3ed1-4d02-85b6-69d180b3f44e service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:35.911518 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618ca03-3ed1-4d02-85b6-69d180b3f44e service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:35.912017 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618ca03-3ed1-4d02-85b6-69d180b3f44e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:35.923333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618ca03-3ed1-4d02-85b6-69d180b3f44e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:35.925751 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618ca03-3ed1-4d02-85b6-69d180b3f44e service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:35.952378 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1295/2568] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:54:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 17:54:36.342763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:36.344625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:36.503544 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a7461-5d91-4e1b-8ec6-536841f75cda None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:36.517526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:36.521553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:36.521553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:36.521553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] PATH_INFO: `/v3/users/b7d18217542243a29f3195c1c9fdac51` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:36.521553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d18217542243a29f3195c1c9fdac51)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:36.532972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:36.737165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d18217542243a29f3195c1c9fdac51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:36.762228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d18217542243a29f3195c1c9fdac51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:36.822654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d18217542243a29f3195c1c9fdac51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:36.823609 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d18217542243a29f3195c1c9fdac51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:36.824352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d18217542243a29f3195c1c9fdac51'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:36.827726 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:36.854339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1274/2569] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:36 2022] DELETE /identity/v3/users/b7d18217542243a29f3195c1c9fdac51 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:36.859412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:36.860009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:36.860009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:36.879614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:36.881264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:36.951190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1296/2570] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:36.963939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:36.968203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:37.061845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0897e4-40cb-4a3a-b151-29833a321748 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:37.067009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:37.067869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:37.068359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:37.068825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:37.082599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:37.084446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:37.096666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1275/2571] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:37.799631 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:37.801309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:37.826470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b40f3d-c7c5-4d9c-88cb-5462632e7318 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:37.870879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:37.871890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:37.872187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:37.874161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] PATH_INFO: `/v3/projects/626b48791ae34848bf321284976c2d72` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:37.877516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=626b48791ae34848bf321284976c2d72)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:37.880260 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:38.065371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626b48791ae34848bf321284976c2d72', 'request_id': 'req-6190448a-f200-48cc-b125-e7a38cd0c8ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.074170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626b48791ae34848bf321284976c2d72', 'request_id': 'req-6190448a-f200-48cc-b125-e7a38cd0c8ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.080389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626b48791ae34848bf321284976c2d72', 'request_id': 'req-6190448a-f200-48cc-b125-e7a38cd0c8ff'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.160320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] The token cache is being invalidate because project 626b48791ae34848bf321284976c2d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:38.160590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.161296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.163013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1297/2572] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:37 2022] DELETE /identity/v3/projects/626b48791ae34848bf321284976c2d72 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:38.167531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:38.183136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbcd9e-91a9-41d6-a5af-4d6f3e99ce2f service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:38.314507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:38.317053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:38.317656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:38.318092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] PATH_INFO: `/v3/users/af2f4fe8e3804710839ca8b9c5be9795` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:38.319834 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2f4fe8e3804710839ca8b9c5be9795)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:38.325404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:38.531604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2f4fe8e3804710839ca8b9c5be9795'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.542922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2f4fe8e3804710839ca8b9c5be9795'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.694054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2f4fe8e3804710839ca8b9c5be9795'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.695660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2f4fe8e3804710839ca8b9c5be9795'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.698026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bec0bf74-e2f6-4bd7-8eef-2052c79c76ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2f4fe8e3804710839ca8b9c5be9795'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:38.718547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1276/2573] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:38 2022] DELETE /identity/v3/users/af2f4fe8e3804710839ca8b9c5be9795 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:39.471215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:39.471215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:39.489615 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190448a-f200-48cc-b125-e7a38cd0c8ff admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:39.561520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:39.562511 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:39.562977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:39.563418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] PATH_INFO: `/v3/projects/c7de33cd22654b7cb0d7fd6f05f5aec2` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:39.564819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7de33cd22654b7cb0d7fd6f05f5aec2)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:39.566864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:39.584883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:39.662793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7de33cd22654b7cb0d7fd6f05f5aec2', 'request_id': 'req-f79657cc-b466-4e7e-9e4a-c3a913471763'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:39.672372 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7de33cd22654b7cb0d7fd6f05f5aec2', 'request_id': 'req-f79657cc-b466-4e7e-9e4a-c3a913471763'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:39.685706 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7de33cd22654b7cb0d7fd6f05f5aec2', 'request_id': 'req-f79657cc-b466-4e7e-9e4a-c3a913471763'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:39.738985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] The token cache is being invalidate because project c7de33cd22654b7cb0d7fd6f05f5aec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:39.739294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:39.739977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f79657cc-b466-4e7e-9e4a-c3a913471763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:39.741583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1298/2574] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:39 2022] DELETE /identity/v3/projects/c7de33cd22654b7cb0d7fd6f05f5aec2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:39.746904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7992-c9ba-4d15-9c33-5203143faee4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:39.747432 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7992-c9ba-4d15-9c33-5203143faee4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:39.747534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7992-c9ba-4d15-9c33-5203143faee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:39.757283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:39.769314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-855a7992-c9ba-4d15-9c33-5203143faee4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:39.770864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-855a7992-c9ba-4d15-9c33-5203143faee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:39.836392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1277/2575] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:39.846948 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cecf-7a29-476a-8951-63f741f0fcef None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:39.848146 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cecf-7a29-476a-8951-63f741f0fcef None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:39.848413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cecf-7a29-476a-8951-63f741f0fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:39.871998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cb69cecf-7a29-476a-8951-63f741f0fcef None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:39.873608 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cb69cecf-7a29-476a-8951-63f741f0fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:39.883079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1299/2576] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:39.887562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7992-c9ba-4d15-9c33-5203143faee4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:39.897795 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7992-c9ba-4d15-9c33-5203143faee4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:39.903453 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:39.904392 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:39.904659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:39.904949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:39.905865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:39.906181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:39.908031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:39.922947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1278/2577] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:39.927523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69cecf-7a29-476a-8951-63f741f0fcef None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:39.938681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69cecf-7a29-476a-8951-63f741f0fcef None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:39.943780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:39.948336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:39.948336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:39.948336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:39.948336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:39.948336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:40.041488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1300/2578] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:39 2022] POST /identity/v3/projects => generated 409 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:40.045784 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:40.057557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac9788-7704-454f-8924-93e6d52977e1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:40.133673 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:40.134678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:40.135123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:40.135572 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:40.136513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:40.138894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:40.235020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1279/2579] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:40 2022] POST /identity/v3/users => generated 305 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:40.236352 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:40.255704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5963d00-2d4f-42cb-9318-89a1ce537d03 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:40.261262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:40.262037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:40.262402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:40.262780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:40.263526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:40.265190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:40.276978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1301/2580] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:40.281401 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:40.303184 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6972fc-ab47-4512-a075-5383e314ed09 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:40.309099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60e159-e072-4382-85bd-596641728e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:40.310228 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60e159-e072-4382-85bd-596641728e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:40.310628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60e159-e072-4382-85bd-596641728e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:40.310975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60e159-e072-4382-85bd-596641728e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:40.311844 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60e159-e072-4382-85bd-596641728e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:40.314168 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60e159-e072-4382-85bd-596641728e6a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:40.335768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1280/2581] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:40.337158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:40.363371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48764b-b269-46d6-b30b-bb88525bad99 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:40.369142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:40.370126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:40.370410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:40.370684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] PATH_INFO: `/v3/projects/89a6e3e1a1dc4e4093ab335dea39f86d/users/3f0998a32b4b4f3c9c0653a28b683f42/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:40.401064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a6e3e1a1dc4e4093ab335dea39f86d, user_id=3f0998a32b4b4f3c9c0653a28b683f42, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:40.404319 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:40.489958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1302/2582] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:40 2022] PUT /identity/v3/projects/89a6e3e1a1dc4e4093ab335dea39f86d/users/3f0998a32b4b4f3c9c0653a28b683f42/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:40.500967 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3637-883b-492b-97c1-eec08dd4f5ab None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:40.501575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3637-883b-492b-97c1-eec08dd4f5ab None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:40.502216 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3637-883b-492b-97c1-eec08dd4f5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:40.559615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-e5fc3637-883b-492b-97c1-eec08dd4f5ab None None] MFA Rules not processed for user `3f0998a32b4b4f3c9c0653a28b683f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:40.559615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-e5fc3637-883b-492b-97c1-eec08dd4f5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:40.647452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:40.652237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1281/2583] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:40.656846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:40.810945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02792f48-28a7-426b-ab94-7b5554bbcbc5 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:40.829072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:40.834422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:40.834753 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:40.835046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:40.847449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:40.847449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:40.952916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1303/2584] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:40 2022] POST /identity/v3/projects => generated 409 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:40.958535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3637-883b-492b-97c1-eec08dd4f5ab None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:40.998566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc3637-883b-492b-97c1-eec08dd4f5ab None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:41.071858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:41.072555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.073113 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.073429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.074100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:41.076147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:41.157680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1282/2585] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:40 2022] POST /identity/v3/users => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:41.163251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:41.187412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed2e32-17d3-4592-90e6-bc23914b02d6 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:41.192341 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:41.197153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.197153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.197153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.197153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:41.207396 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:41.239744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1304/2586] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:41.246664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:41.275361 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd858d-1c2b-41da-b468-441bd83be045 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:41.281007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:41.282309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.282804 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.284372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] PATH_INFO: `/v3/projects/613b33d1ffd446408ea05d7b32d52549/users/d1b88b4efe9a4a42b5514426c9ebd176/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.309538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=613b33d1ffd446408ea05d7b32d52549, user_id=d1b88b4efe9a4a42b5514426c9ebd176, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:41.312618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:41.384305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1283/2587] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:41 2022] PUT /identity/v3/projects/613b33d1ffd446408ea05d7b32d52549/users/d1b88b4efe9a4a42b5514426c9ebd176/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:41.385946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:41.405189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbf403-4340-4269-87d5-7c5e19f485a2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:41.434216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:41.505125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44857f87-4716-49b4-9080-ee307e4c78f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:41.506223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44857f87-4716-49b4-9080-ee307e4c78f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.506967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44857f87-4716-49b4-9080-ee307e4c78f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.507526 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44857f87-4716-49b4-9080-ee307e4c78f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.508439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44857f87-4716-49b4-9080-ee307e4c78f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:41.510205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44857f87-4716-49b4-9080-ee307e4c78f0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:41.522662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1305/2588] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:41.527813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:41.542777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb74590-0a58-43e3-8194-de9b953b99cd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:41.549791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:41.551153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.559683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.559683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] PATH_INFO: `/v3/projects/613b33d1ffd446408ea05d7b32d52549/users/d1b88b4efe9a4a42b5514426c9ebd176/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.559683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613b33d1ffd446408ea05d7b32d52549, user_id=d1b88b4efe9a4a42b5514426c9ebd176, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:41.561658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:41.577660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1284/2589] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:41 2022] PUT /identity/v3/projects/613b33d1ffd446408ea05d7b32d52549/users/d1b88b4efe9a4a42b5514426c9ebd176/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:41.590553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.591084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.591606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.624390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] MFA Rules not processed for user `d1b88b4efe9a4a42b5514426c9ebd176`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:41.625572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:41.700714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1306/2590] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:41 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:41.712343 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:41.716244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:41.801366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb73a-4bbe-46cd-affd-aedfdeb87208 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:41.805434 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:41.806166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.806539 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.806888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.819558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:41.819558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:41.847055 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1285/2591] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:41 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:41.896793 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:41.898603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:41.989082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f977d-ac9a-4eda-9bc1-65437ae49f82 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:41.992924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:41.993589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:41.993972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:41.994306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:41.996188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:42.075749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:42.092988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:42.094553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:42.106474 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1307/2592] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:43.741320 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:43.751902 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3f348-5123-4e8b-a4e6-34d2738365f7 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:43.764643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:43.765322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:43.765614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:43.765899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:43.872991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:43.886131 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:43.887670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:43.914293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1286/2593] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:46.720898 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:46.723132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:46.737279 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70d40d-f291-49ba-95af-7a832dd9cd8e service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:46.743869 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:46.744941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:46.745699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:46.745699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] PATH_INFO: `/v3/users/d9a8fcc207544205bec6ffa95069ec7e` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:46.746725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a8fcc207544205bec6ffa95069ec7e)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:46.749250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:46.820889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:46.822527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:46.844126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833090f-3cf9-41a3-895a-42ea3233c811 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:46.848942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:46.849854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:46.850142 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:46.850417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] PATH_INFO: `/v3/users/caab1412244c43adac226cb253a10880` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:46.851790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caab1412244c43adac226cb253a10880)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:46.853735 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:46.908125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a8fcc207544205bec6ffa95069ec7e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:46.931389 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a8fcc207544205bec6ffa95069ec7e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:46.933392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a8fcc207544205bec6ffa95069ec7e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:46.941973 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a8fcc207544205bec6ffa95069ec7e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:46.942171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-84c2543a-e84b-4b86-8a4a-38950bc1d54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a8fcc207544205bec6ffa95069ec7e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.006524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1308/2594] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:46 2022] DELETE /identity/v3/users/d9a8fcc207544205bec6ffa95069ec7e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:47.008554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.008907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.009223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.027533 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:47.027837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caab1412244c43adac226cb253a10880'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.028789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:47.035707 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caab1412244c43adac226cb253a10880'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.076211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caab1412244c43adac226cb253a10880'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.076861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caab1412244c43adac226cb253a10880'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.077477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caab1412244c43adac226cb253a10880'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.090772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1309/2595] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:47.096024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1287/2596] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:46 2022] DELETE /identity/v3/users/caab1412244c43adac226cb253a10880 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:47.100556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:47.102040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:47.162462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b728d2f-af2b-40d5-8536-71a60836e8f2 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:47.175014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:47.216230 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad12ce-dc58-4e8c-965c-d1e71134e821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:47.216992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad12ce-dc58-4e8c-965c-d1e71134e821 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.217611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad12ce-dc58-4e8c-965c-d1e71134e821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.217901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad12ce-dc58-4e8c-965c-d1e71134e821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.286681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad12ce-dc58-4e8c-965c-d1e71134e821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:47.288130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad12ce-dc58-4e8c-965c-d1e71134e821 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:47.298151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1310/2597] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:47.346305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:47.350886 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:47.367595 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67f42b-74fc-4dc2-8d26-cb740317ea01 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:47.372935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:47.374163 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.374469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.374738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] PATH_INFO: `/v3/projects/14a00a57ce8e46f0b0936cbe9c6ae9d8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.375987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a00a57ce8e46f0b0936cbe9c6ae9d8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:47.382985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:47.500404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a00a57ce8e46f0b0936cbe9c6ae9d8', 'request_id': 'req-2c125847-e4ea-4a50-ba5a-6319d6ed9601'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.509255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a00a57ce8e46f0b0936cbe9c6ae9d8', 'request_id': 'req-2c125847-e4ea-4a50-ba5a-6319d6ed9601'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.596335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a00a57ce8e46f0b0936cbe9c6ae9d8', 'request_id': 'req-2c125847-e4ea-4a50-ba5a-6319d6ed9601'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.608755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] The token cache is being invalidate because project 14a00a57ce8e46f0b0936cbe9c6ae9d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:47.609053 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.609694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-2c125847-e4ea-4a50-ba5a-6319d6ed9601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.611258 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1288/2598] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:47 2022] DELETE /identity/v3/projects/14a00a57ce8e46f0b0936cbe9c6ae9d8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:47.618018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee240cf6-70aa-483d-9f16-0189eb1c40c0 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.618313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee240cf6-70aa-483d-9f16-0189eb1c40c0 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.618596 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee240cf6-70aa-483d-9f16-0189eb1c40c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.636731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ee240cf6-70aa-483d-9f16-0189eb1c40c0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:47.638062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ee240cf6-70aa-483d-9f16-0189eb1c40c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:47.699267 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1311/2599] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:47.707104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.707397 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.709265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.726623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:47.728056 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:47.735896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1289/2600] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:47.740071 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee240cf6-70aa-483d-9f16-0189eb1c40c0 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:47.750768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee240cf6-70aa-483d-9f16-0189eb1c40c0 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:47.755310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:47.755961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.756245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.756520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.757313 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:47.757599 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:47.759201 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:47.772182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:47.774938 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1312/2601] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:47.779364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:47.784353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:47.791786 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb825c3-df79-46cd-86c4-62c6d2a0a4eb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:47.798014 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:47.798685 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.798967 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.799245 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.800088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:47.803315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:47.817932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0b0ff-980a-4aee-9479-283c77ade4ed None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:47.829659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:47.831486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:47.831988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:47.832627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] PATH_INFO: `/v3/projects/c9d18877754d4c70a02358d0cee5b13e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:47.833893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d18877754d4c70a02358d0cee5b13e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:47.835939 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:47.887435 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1313/2602] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:47 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:47.892027 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:47.933992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:47.941520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:47.953714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d18877754d4c70a02358d0cee5b13e', 'request_id': 'req-855241ed-1a64-44a3-92dd-ecd7b2433387'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.962263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d18877754d4c70a02358d0cee5b13e', 'request_id': 'req-855241ed-1a64-44a3-92dd-ecd7b2433387'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:47.971088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604c11b-b7fa-4a83-b6b1-b2fe4d88a45a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.025016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d18877754d4c70a02358d0cee5b13e', 'request_id': 'req-855241ed-1a64-44a3-92dd-ecd7b2433387'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:48.032057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.032768 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.033053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.033330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.034019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.035953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.036390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] The token cache is being invalidate because project c9d18877754d4c70a02358d0cee5b13e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:48.036712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:48.037366 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-855241ed-1a64-44a3-92dd-ecd7b2433387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:48.039590 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1290/2603] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:47 2022] DELETE /identity/v3/projects/c9d18877754d4c70a02358d0cee5b13e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:48.044295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699426a2-48ce-44c9-b1e9-0dff0a2cbd57 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.044586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699426a2-48ce-44c9-b1e9-0dff0a2cbd57 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.044890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699426a2-48ce-44c9-b1e9-0dff0a2cbd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.062930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-699426a2-48ce-44c9-b1e9-0dff0a2cbd57 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:48.064130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-699426a2-48ce-44c9-b1e9-0dff0a2cbd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:48.072571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1291/2604] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:48.080342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac978-92a0-4af6-afa5-9a751f2281d4 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.080689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac978-92a0-4af6-afa5-9a751f2281d4 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.080983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac978-92a0-4af6-afa5-9a751f2281d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.099480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-302ac978-92a0-4af6-afa5-9a751f2281d4 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:48.100677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-302ac978-92a0-4af6-afa5-9a751f2281d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:48.109377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1292/2605] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:48.111787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1314/2606] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:47 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:48.116157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.116303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ac978-92a0-4af6-afa5-9a751f2281d4 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.117999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:48.125662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ac978-92a0-4af6-afa5-9a751f2281d4 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.131179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.131356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15758a15-c141-4ad7-b931-0443b112be3e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.131842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.132202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.132554 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.133250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:48.133599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.135224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.136634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.137571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.138008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.138287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.139011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.140616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.147417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1293/2607] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:48.151351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1315/2608] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:48.152502 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.154998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.162522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675db097-e8da-4e66-8959-cd048df50742 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.165806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bff82-ba3b-4e42-a780-47aacacdb902 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.167772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.168417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.168738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.169019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.169704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.170774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.171424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.171613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.171719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.171975 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.172735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.174305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.186147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1316/2609] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:48.190025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.200565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3fc4e-4a12-4de6-8af5-c8e5691fb624 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.205221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029db9a-e8a2-4b93-bb2a-54fb74021e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.207558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029db9a-e8a2-4b93-bb2a-54fb74021e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.207558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029db9a-e8a2-4b93-bb2a-54fb74021e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.207558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029db9a-e8a2-4b93-bb2a-54fb74021e7c admin admin] PATH_INFO: `/v3/projects/e1b19341f7b94efa925576c2deb06bc1/users/6e1cbf61b93d4eeaaea404e1764aee45/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.232067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029db9a-e8a2-4b93-bb2a-54fb74021e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b19341f7b94efa925576c2deb06bc1, user_id=6e1cbf61b93d4eeaaea404e1764aee45, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.236713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029db9a-e8a2-4b93-bb2a-54fb74021e7c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.254117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1294/2610] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:48.254117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.316221 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389900b-2be6-41c4-b757-764147e1c15f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.321522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.322196 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.322484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.322761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.323506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.325469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.335373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1317/2611] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:48 2022] PUT /identity/v3/projects/e1b19341f7b94efa925576c2deb06bc1/users/6e1cbf61b93d4eeaaea404e1764aee45/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:48.345147 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd0eb-e339-43d6-bfb3-fd725f9d0bd3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.345442 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd0eb-e339-43d6-bfb3-fd725f9d0bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.345717 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd0eb-e339-43d6-bfb3-fd725f9d0bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.372132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-49dcd0eb-e339-43d6-bfb3-fd725f9d0bd3 None None] MFA Rules not processed for user `6e1cbf61b93d4eeaaea404e1764aee45`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:48.373370 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-49dcd0eb-e339-43d6-bfb3-fd725f9d0bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:48.413460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1295/2612] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:48.417645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.427195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799c494-fb4c-4137-9e1e-336b3e40f186 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.440574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1318/2613] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:48.445385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd0eb-e339-43d6-bfb3-fd725f9d0bd3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.457218 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:48.485796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.486488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.486773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.487057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.487793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.489452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.499893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1296/2614] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:48.504067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.509684 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:48.511419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd0eb-e339-43d6-bfb3-fd725f9d0bd3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.515420 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc492e89-03b1-487c-a04c-62b26a414b84 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.516099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.517084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.517790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.518063 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.518758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.520708 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.521475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.522336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.524459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.524459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.524459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.526825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.537945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1297/2615] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:48.542016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.551799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961b302-65d4-439f-bf07-ab25943d1efd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.556460 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f5623-52e2-430e-a834-2e5e846844dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.557362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f5623-52e2-430e-a834-2e5e846844dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.557652 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f5623-52e2-430e-a834-2e5e846844dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.557935 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f5623-52e2-430e-a834-2e5e846844dc admin admin] PATH_INFO: `/v3/projects/1e37964d56b545a991b28be9a4f997f1/users/046d0f5ade9e41e7bfd913b08d933166/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.572568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f5623-52e2-430e-a834-2e5e846844dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e37964d56b545a991b28be9a4f997f1, user_id=046d0f5ade9e41e7bfd913b08d933166, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.575294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f5623-52e2-430e-a834-2e5e846844dc admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.609249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1319/2616] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:48.613312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.676677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1298/2617] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:48 2022] PUT /identity/v3/projects/1e37964d56b545a991b28be9a4f997f1/users/046d0f5ade9e41e7bfd913b08d933166/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:48.678421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000447-2e13-43b6-9228-7e971ae9967d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.688686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e9bd-6086-4bf2-8f41-62b38ad2704f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.688924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e9bd-6086-4bf2-8f41-62b38ad2704f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.689201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e9bd-6086-4bf2-8f41-62b38ad2704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.715310 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9ec0e9bd-6086-4bf2-8f41-62b38ad2704f None None] MFA Rules not processed for user `046d0f5ade9e41e7bfd913b08d933166`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:48.716525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9ec0e9bd-6086-4bf2-8f41-62b38ad2704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:48.736959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.737633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.737917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.738192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.738882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.740892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.782235 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1299/2618] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:48.825096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e9bd-6086-4bf2-8f41-62b38ad2704f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.825096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e9bd-6086-4bf2-8f41-62b38ad2704f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.825096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.825096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.825526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.825526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.825526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.825526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.922402 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1320/2619] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/users => generated 296 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:48.926266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.936424 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767a8a6-858b-4ca0-b1bd-950a13b4b871 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.942009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:48.942649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:48.942936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:48.943220 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:48.943947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:48.946249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:48.957169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1321/2620] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:48.957746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1300/2621] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:48.961377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.961516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:48.971076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaf318-bb66-4ca5-9e34-1ea8b164961c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:48.990136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:49.026052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a705b-37ed-4a65-8d9d-02287a25d241 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.033522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.034667 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.035080 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.035467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.035908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.036865 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.036998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.037360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.037651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] PATH_INFO: `/v3/projects/631ad83a77244fb09f4cb6e268f41a35/users/c5d08a70e3b24c53b68fea18ce7cf21d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.044883 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.056345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631ad83a77244fb09f4cb6e268f41a35, user_id=c5d08a70e3b24c53b68fea18ce7cf21d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.059788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.124623 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1301/2622] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:48 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:49.127144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.137627 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8a6a-b6b8-472f-bbc5-bd35a3e23167 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.142357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.143026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.143381 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.143668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.144408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.146082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.152779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1322/2623] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:48 2022] PUT /identity/v3/projects/631ad83a77244fb09f4cb6e268f41a35/users/c5d08a70e3b24c53b68fea18ce7cf21d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:49.157139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.157632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1302/2624] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:49.161750 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.172282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59368-ad97-4329-bfa7-4b563a012812 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.182682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:49.229188 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a046eb0-fe7a-4617-8002-44abb4463296 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.234154 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.234828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.235120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.235404 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.236138 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.237286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.237792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.238173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.238457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.238733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] PATH_INFO: `/v3/projects/7e3635d49c9b4c619156e1b84ce60ebf/users/c39001c188ac4acda8557b797f2e9403/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.249467 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1323/2625] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:49.253725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.255731 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3635d49c9b4c619156e1b84ce60ebf, user_id=c39001c188ac4acda8557b797f2e9403, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.258556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.263679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1e050-56c5-43e5-a99c-db495254dfcf admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.268470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e86b15-85c5-4586-9b02-173ba133c798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.269384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e86b15-85c5-4586-9b02-173ba133c798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.269669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e86b15-85c5-4586-9b02-173ba133c798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.269945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e86b15-85c5-4586-9b02-173ba133c798 admin admin] PATH_INFO: `/v3/projects/631ad83a77244fb09f4cb6e268f41a35/users/c5d08a70e3b24c53b68fea18ce7cf21d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.271993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e86b15-85c5-4586-9b02-173ba133c798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631ad83a77244fb09f4cb6e268f41a35, user_id=c5d08a70e3b24c53b68fea18ce7cf21d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.274733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e86b15-85c5-4586-9b02-173ba133c798 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.284923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1324/2626] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:49 2022] PUT /identity/v3/projects/631ad83a77244fb09f4cb6e268f41a35/users/c5d08a70e3b24c53b68fea18ce7cf21d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:49.296163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.296458 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.296847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.323005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] MFA Rules not processed for user `c5d08a70e3b24c53b68fea18ce7cf21d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:49.326345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:49.351334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1303/2627] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:49 2022] PUT /identity/v3/projects/7e3635d49c9b4c619156e1b84ce60ebf/users/c39001c188ac4acda8557b797f2e9403/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:49.355191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.395763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1325/2628] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:49.408297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.410126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:49.418575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a76622-29ac-4c86-8b34-0070cc228230 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.423330 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.423987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.424280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.424575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.424974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:49.425323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.426956 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.437618 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1304/2629] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:49.441891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.451828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be829b49-ec5d-4d82-bba3-9fde40ed9708 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.456714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726655e-960f-49b9-85fb-33ae9822b8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.460816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726655e-960f-49b9-85fb-33ae9822b8b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.460816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726655e-960f-49b9-85fb-33ae9822b8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.460816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726655e-960f-49b9-85fb-33ae9822b8b8 admin admin] PATH_INFO: `/v3/projects/7e3635d49c9b4c619156e1b84ce60ebf/users/c39001c188ac4acda8557b797f2e9403/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.460816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726655e-960f-49b9-85fb-33ae9822b8b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3635d49c9b4c619156e1b84ce60ebf, user_id=c39001c188ac4acda8557b797f2e9403, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.463507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726655e-960f-49b9-85fb-33ae9822b8b8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.471614 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca112df-4cf2-4ceb-ab73-50e981aa0733 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.474350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1305/2630] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:49 2022] PUT /identity/v3/projects/7e3635d49c9b4c619156e1b84ce60ebf/users/c39001c188ac4acda8557b797f2e9403/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:49.485567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.486047 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.486331 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.518721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] MFA Rules not processed for user `c39001c188ac4acda8557b797f2e9403`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:49.519948 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:49.550461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.551285 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.551673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.552056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.627374 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1306/2631] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:49.629867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.631321 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.642508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.644315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:49.651315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1326/2632] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:49.654552 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:49.658994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65fcc4-2e47-4150-8812-126c2728596f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.662941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.663798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.664431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.664431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.674831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.676292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.687079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1307/2633] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:54:49.844367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:49.853303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3863e-f3dc-4c3b-a4a7-be0280f4325d service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:49.857554 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:49.858233 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:49.858520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:49.858798 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:49.918936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:49.920307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:49.929448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1327/2634] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:50.684135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:50.695591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cd9ba-f743-4631-a1fe-5f3bda913fcd service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:50.700719 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5d08a70e3b24c53b68fea18ce7cf21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631ad83a77244fb09f4cb6e268f41a35', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:50.701467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:50.701761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:50.702245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:50.703212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:50.703690 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:50.705513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:50.720645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1308/2635] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:50.725339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:50.735588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbdef0-e343-4a4f-876f-a15f78491d56 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:50.740409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5d08a70e3b24c53b68fea18ce7cf21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631ad83a77244fb09f4cb6e268f41a35', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:50.741118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:50.741468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:50.742332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:50.743116 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:50.745077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:50.759521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:50.877522 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1328/2636] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:50 2022] POST /identity/v3/projects => generated 354 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:50.879610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:50.892810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b08269-0ae7-483d-8bf7-54956de14fa2 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:50.952840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5d08a70e3b24c53b68fea18ce7cf21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631ad83a77244fb09f4cb6e268f41a35', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:50.956187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:50.956187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:50.956187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:50.956187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:50.956187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:50.956791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:50.970720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1309/2637] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:51.027721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:51.047005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038991a1-cdbd-4979-b5ee-153a01191d23 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:51.053638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5d08a70e3b24c53b68fea18ce7cf21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '631ad83a77244fb09f4cb6e268f41a35', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:51.053638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:51.053638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:51.053971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] PATH_INFO: `/v3/projects/583a237fb4a742f288df60fb67d87fbd` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:51.054885 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorizing `identity:delete_project(project_id=583a237fb4a742f288df60fb67d87fbd)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:51.056870 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:51.164249 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583a237fb4a742f288df60fb67d87fbd', 'request_id': 'req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:51.170886 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583a237fb4a742f288df60fb67d87fbd', 'request_id': 'req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:51.176646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583a237fb4a742f288df60fb67d87fbd', 'request_id': 'req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:51.236314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] The token cache is being invalidate because project 583a237fb4a742f288df60fb67d87fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:51.236625 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:51.237318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-53cc490e-6b6f-4cec-a3b9-ddffbf44c7f7 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:51.238895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1329/2638] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:51 2022] DELETE /identity/v3/projects/583a237fb4a742f288df60fb67d87fbd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:51.852495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:51.854227 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:51.927373 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36d53a-3389-4be9-8047-39b4d2ddb280 tempest-VolumeQuotaClassesTest-612358899 tempest-VolumeQuotaClassesTest-612358899-project-admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:51.989078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:51.989882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:51.990165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:51.990438 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] PATH_INFO: `/v3/users/6e1cbf61b93d4eeaaea404e1764aee45` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:51.991628 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1cbf61b93d4eeaaea404e1764aee45)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:51.993678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:52.016790 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:52.176588 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1cbf61b93d4eeaaea404e1764aee45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:52.183752 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1cbf61b93d4eeaaea404e1764aee45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:52.241928 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1cbf61b93d4eeaaea404e1764aee45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:52.242622 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1cbf61b93d4eeaaea404e1764aee45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:52.243250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1cbf61b93d4eeaaea404e1764aee45'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:52.266723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1310/2639] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:51 2022] DELETE /identity/v3/users/6e1cbf61b93d4eeaaea404e1764aee45 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:52.271110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:52.271638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:52.271737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:52.289653 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:52.290872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:52.364937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1330/2640] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:52.371858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:52.373341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:52.433411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679790f3-4b45-4ea2-bf03-117fcd372cb0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:52.437059 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:52.437711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:52.438002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:52.438287 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:52.447470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:52.452085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:52.458370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1311/2641] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:52.913695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:52.915179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:52.932357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81221b-63a9-411f-809e-73ff84e8dd87 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:52.944395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:52.945889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:52.946356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:52.946635 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] PATH_INFO: `/v3/projects/e1b19341f7b94efa925576c2deb06bc1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:52.948182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b19341f7b94efa925576c2deb06bc1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:52.950085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:53.089413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b19341f7b94efa925576c2deb06bc1', 'request_id': 'req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.095923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b19341f7b94efa925576c2deb06bc1', 'request_id': 'req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.102094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b19341f7b94efa925576c2deb06bc1', 'request_id': 'req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.184984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] The token cache is being invalidate because project e1b19341f7b94efa925576c2deb06bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:53.185274 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.185901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.187583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1331/2642] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:52 2022] DELETE /identity/v3/projects/e1b19341f7b94efa925576c2deb06bc1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:53.192015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:53.202306 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75826-00c1-423c-b4dd-8c7d4fd070c4 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:53.261408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:53.262372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:53.262662 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:53.262945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] PATH_INFO: `/v3/users/c5d08a70e3b24c53b68fea18ce7cf21d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:53.264141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d08a70e3b24c53b68fea18ce7cf21d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:53.266148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:53.460545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:53.464958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d08a70e3b24c53b68fea18ce7cf21d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.473648 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d08a70e3b24c53b68fea18ce7cf21d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.527551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d08a70e3b24c53b68fea18ce7cf21d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.528566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d08a70e3b24c53b68fea18ce7cf21d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.529309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-b34b8395-65cd-4a8f-a8c5-50d019475415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d08a70e3b24c53b68fea18ce7cf21d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:53.548016 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1312/2643] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:53 2022] DELETE /identity/v3/users/c5d08a70e3b24c53b68fea18ce7cf21d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:54.050187 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.053383 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:54.065974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:54.078069 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebce1f3-0a7c-4e25-b1b2-93b770269d84 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.142291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.143253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.143523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.144042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] PATH_INFO: `/v3/projects/631ad83a77244fb09f4cb6e268f41a35` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.146118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631ad83a77244fb09f4cb6e268f41a35)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.148395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.274780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631ad83a77244fb09f4cb6e268f41a35', 'request_id': 'req-898b45ab-28f0-48f1-83c8-6b23967b7849'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.281845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631ad83a77244fb09f4cb6e268f41a35', 'request_id': 'req-898b45ab-28f0-48f1-83c8-6b23967b7849'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.289234 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631ad83a77244fb09f4cb6e268f41a35', 'request_id': 'req-898b45ab-28f0-48f1-83c8-6b23967b7849'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.340052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] The token cache is being invalidate because project 631ad83a77244fb09f4cb6e268f41a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:54.340340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.341010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-898b45ab-28f0-48f1-83c8-6b23967b7849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.342591 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1332/2644] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:54 2022] DELETE /identity/v3/projects/631ad83a77244fb09f4cb6e268f41a35 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:54.347947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb844674-f817-45e9-802b-31df4c7dde4d None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.348495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb844674-f817-45e9-802b-31df4c7dde4d None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.348495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb844674-f817-45e9-802b-31df4c7dde4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.367642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-fb844674-f817-45e9-802b-31df4c7dde4d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:54.368920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fb844674-f817-45e9-802b-31df4c7dde4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:54.433470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1313/2645] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:54.440978 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeca521-6836-4d22-9f73-64b073afb265 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.441275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeca521-6836-4d22-9f73-64b073afb265 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.441557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeca521-6836-4d22-9f73-64b073afb265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.462310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dfeca521-6836-4d22-9f73-64b073afb265 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:54.463535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dfeca521-6836-4d22-9f73-64b073afb265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:54.471791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1333/2646] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:54.476125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb844674-f817-45e9-802b-31df4c7dde4d None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.485806 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb844674-f817-45e9-802b-31df4c7dde4d None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.490599 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.491309 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.491611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.491894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.492660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:54.492975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.494619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.513974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1314/2647] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:54.514145 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeca521-6836-4d22-9f73-64b073afb265 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.522297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeca521-6836-4d22-9f73-64b073afb265 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.527066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.527721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.528011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.528293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.529024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.530941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.567987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.570230 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:54.583555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580553f1-72d7-4356-8262-f5e339823e65 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.589050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.589920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.590205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.590481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] PATH_INFO: `/v3/users/3f0998a32b4b4f3c9c0653a28b683f42` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.591670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0998a32b4b4f3c9c0653a28b683f42)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.593683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.633228 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1334/2648] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:54 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:54.637351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.705191 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5360e4f-8ef8-4a26-8fc2-be6a7bcb661b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.710066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.710739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.711025 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.711306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.712008 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.714130 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.722958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:54.830304 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0998a32b4b4f3c9c0653a28b683f42'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.837586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0998a32b4b4f3c9c0653a28b683f42'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.849156 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1335/2649] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:54 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:54.853330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.863090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ff196-ff30-4e13-9039-f402b60a0b60 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.868452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.869142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.869432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.869711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.870444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.872053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.888500 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1336/2650] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:54.889553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.899342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0374859-919a-4829-91b3-32890c309f9c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.904152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.904836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.905134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.905418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.906151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.907735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.920927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1337/2651] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:54.922013 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:54.922576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0998a32b4b4f3c9c0653a28b683f42'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.923378 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0998a32b4b4f3c9c0653a28b683f42'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.924072 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-112d0abc-a2d5-4131-850d-c5f530e0dec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0998a32b4b4f3c9c0653a28b683f42'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:54.932902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17071098-21cb-4c08-a406-3349d49cf96c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:54.939829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:54.940838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.941131 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.941416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] PATH_INFO: `/v3/projects/24d6f3d46ba74bf087e4b75741de86d8/users/64e638d550a7429c8e91b64513a47c20/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.950733 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1315/2652] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:54 2022] DELETE /identity/v3/users/3f0998a32b4b4f3c9c0653a28b683f42 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:54.952451 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bf34a-3b93-4318-9cbb-c501b8d70a62 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:54.953061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bf34a-3b93-4318-9cbb-c501b8d70a62 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:54.953061 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bf34a-3b93-4318-9cbb-c501b8d70a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:54.957330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d6f3d46ba74bf087e4b75741de86d8, user_id=64e638d550a7429c8e91b64513a47c20, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:54.960144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:54.971607 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-020bf34a-3b93-4318-9cbb-c501b8d70a62 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:54.974182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-020bf34a-3b93-4318-9cbb-c501b8d70a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:54.985800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:55.035028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1316/2653] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:55.036665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1338/2654] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:54 2022] PUT /identity/v3/projects/24d6f3d46ba74bf087e4b75741de86d8/users/64e638d550a7429c8e91b64513a47c20/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:55.048121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.048387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.048768 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.049006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.049883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:55.078991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] MFA Rules not processed for user `64e638d550a7429c8e91b64513a47c20`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:55.080218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:55.107945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed6ed7-c99c-4eb4-8519-70960047ed8c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.111710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.112351 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.112665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.112951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.144315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1317/2655] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:55.148958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.150352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:55.179153 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.180744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.191310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1339/2656] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:54:55.221828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd50f2-a470-40f9-85d5-6f4a703613f6 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.230876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.231563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.231849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.232164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.232964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.237981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.376513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1318/2657] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:55 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:54:55.380490 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.391021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3158a-3050-4ca5-b6ee-59ae8a93939d service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.443931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:55.450353 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.451035 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.451332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.451936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.452310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.454286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.565133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1340/2658] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:55 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:55.566368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.577311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e4ace-7c3e-48aa-8acc-53d885cd5ebe admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.582222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.582891 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.589895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.589895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.589895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.589895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.599907 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1319/2659] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:55.610662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.621665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be26ba-1241-431d-aeab-5e5d4420ca57 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.626680 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.627562 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.627796 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.628101 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] PATH_INFO: `/v3/projects/952700b80a394e94a0400330e994c8f3/users/7e01c4a1795c4f4980d4cf12a598c9c1/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.644255 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952700b80a394e94a0400330e994c8f3, user_id=7e01c4a1795c4f4980d4cf12a598c9c1, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.646962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.663547 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.665463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:55.682060 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47342f-f038-4889-9e1e-9840ebd73a34 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.686916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.687745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.688543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.688543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] PATH_INFO: `/v3/projects/89a6e3e1a1dc4e4093ab335dea39f86d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.690203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a6e3e1a1dc4e4093ab335dea39f86d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.692480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.716192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:55.739722 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1341/2660] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:55 2022] PUT /identity/v3/projects/952700b80a394e94a0400330e994c8f3/users/7e01c4a1795c4f4980d4cf12a598c9c1/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:55.743992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.803791 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a6e3e1a1dc4e4093ab335dea39f86d', 'request_id': 'req-a84b59a9-e568-46cb-967c-3ee6fba1dd02'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:55.806261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1f9e-75aa-41fa-b92d-d2338185c797 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.810703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a6e3e1a1dc4e4093ab335dea39f86d', 'request_id': 'req-a84b59a9-e568-46cb-967c-3ee6fba1dd02'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:55.864225 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.864924 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.865324 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.865641 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.866430 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.868037 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.880031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1342/2661] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:55.883081 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.892797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59810a79-0032-4bd4-bcb1-4a1136fa9f81 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.897647 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b08a-2faf-4abf-9e19-e49a3b32381e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.899011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b08a-2faf-4abf-9e19-e49a3b32381e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.899298 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b08a-2faf-4abf-9e19-e49a3b32381e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.899578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b08a-2faf-4abf-9e19-e49a3b32381e admin admin] PATH_INFO: `/v3/projects/952700b80a394e94a0400330e994c8f3/users/7e01c4a1795c4f4980d4cf12a598c9c1/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.899799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a6e3e1a1dc4e4093ab335dea39f86d', 'request_id': 'req-a84b59a9-e568-46cb-967c-3ee6fba1dd02'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:55.901628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b08a-2faf-4abf-9e19-e49a3b32381e admin admin] RBAC: Authorizing `identity:create_grant(project_id=952700b80a394e94a0400330e994c8f3, user_id=7e01c4a1795c4f4980d4cf12a598c9c1, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.904395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b08a-2faf-4abf-9e19-e49a3b32381e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.911043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] The token cache is being invalidate because project 89a6e3e1a1dc4e4093ab335dea39f86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:55.911339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:55.911988 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:55.913602 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1320/2662] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:55 2022] DELETE /identity/v3/projects/89a6e3e1a1dc4e4093ab335dea39f86d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:55.916569 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1343/2663] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:55 2022] PUT /identity/v3/projects/952700b80a394e94a0400330e994c8f3/users/7e01c4a1795c4f4980d4cf12a598c9c1/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:55.917509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:55.918890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:55.926235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.926628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.927084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.935003 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b59a9-e568-46cb-967c-3ee6fba1dd02 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:55.955076 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] MFA Rules not processed for user `7e01c4a1795c4f4980d4cf12a598c9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:55.956308 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:55.992505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:55.993486 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:55.993769 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:55.994047 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] PATH_INFO: `/v3/users/d1b88b4efe9a4a42b5514426c9ebd176` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:55.995252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b88b4efe9a4a42b5514426c9ebd176)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:55.997237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:55.998259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:56.034084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1344/2664] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:54:56.045327 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:56.046649 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:56.103283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1583f-1559-4997-a5c8-12831a47d0b8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:56.107057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:56.107709 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:56.107996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:56.108578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:56.118086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:56.119560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:56.129842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1345/2665] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:54:56.168117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b88b4efe9a4a42b5514426c9ebd176'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.172563 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b88b4efe9a4a42b5514426c9ebd176'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.229727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:56.235927 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b88b4efe9a4a42b5514426c9ebd176'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.236571 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b88b4efe9a4a42b5514426c9ebd176'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.237220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b88b4efe9a4a42b5514426c9ebd176'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.238689 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676b3cc-da8f-4694-b3e6-75468f23a0b5 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:56.242360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e588c8-5fc3-4730-84cb-ad5097da9811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:56.243002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e588c8-5fc3-4730-84cb-ad5097da9811 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:56.243286 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e588c8-5fc3-4730-84cb-ad5097da9811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:56.243563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e588c8-5fc3-4730-84cb-ad5097da9811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:56.245368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-e1e588c8-5fc3-4730-84cb-ad5097da9811 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:56.262164 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1321/2666] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:55 2022] DELETE /identity/v3/users/d1b88b4efe9a4a42b5514426c9ebd176 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:54:56.309810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e588c8-5fc3-4730-84cb-ad5097da9811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:56.311299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e588c8-5fc3-4730-84cb-ad5097da9811 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:56.320743 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1346/2667] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:54:56.677893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:56.677893 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:56.741473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:56.776465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d2115-f438-4165-acdc-95b9452beb10 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:56.781141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:56.782011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:56.782301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:56.782584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] PATH_INFO: `/v3/projects/613b33d1ffd446408ea05d7b32d52549` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:56.783794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613b33d1ffd446408ea05d7b32d52549)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:56.785761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:56.870917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613b33d1ffd446408ea05d7b32d52549', 'request_id': 'req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.877352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613b33d1ffd446408ea05d7b32d52549', 'request_id': 'req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.956714 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613b33d1ffd446408ea05d7b32d52549', 'request_id': 'req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.968468 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] The token cache is being invalidate because project 613b33d1ffd446408ea05d7b32d52549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:54:56.968819 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.969456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c15926c7-5d4f-4fce-a60c-fdd6d2779e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:54:56.977334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1322/2668] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:54:56 2022] DELETE /identity/v3/projects/613b33d1ffd446408ea05d7b32d52549 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:56.982258 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625323f-17cc-493a-aec4-d1edfe6c1311 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:56.984039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625323f-17cc-493a-aec4-d1edfe6c1311 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:56.984039 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625323f-17cc-493a-aec4-d1edfe6c1311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.001541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1625323f-17cc-493a-aec4-d1edfe6c1311 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:57.002810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1625323f-17cc-493a-aec4-d1edfe6c1311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:57.076583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1347/2669] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:57.084172 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7104a-4858-40ec-84f6-4003ea349002 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.084465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7104a-4858-40ec-84f6-4003ea349002 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.084786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7104a-4858-40ec-84f6-4003ea349002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.104194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-9be7104a-4858-40ec-84f6-4003ea349002 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:57.105538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9be7104a-4858-40ec-84f6-4003ea349002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:57.113921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1323/2670] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:57.118912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625323f-17cc-493a-aec4-d1edfe6c1311 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:57.131082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625323f-17cc-493a-aec4-d1edfe6c1311 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:57.136732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:57.137382 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.137665 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.137937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.138790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:54:57.139094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:57.140715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:57.152847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1348/2671] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:54:57.157136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7104a-4858-40ec-84f6-4003ea349002 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:57.166583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7104a-4858-40ec-84f6-4003ea349002 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:57.171773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:57.172558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.172882 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.173167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.173872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:57.175895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:57.283851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1324/2672] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:57 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:57.288995 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:57.299062 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c9e02-2184-4e66-990d-352019654fe8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:57.302721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:57.364170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:57.364977 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.365275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.365560 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.367271 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:57.369532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:57.470767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1349/2673] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:57 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:57.474862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:57.485812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1fdfe-68a4-435c-992c-92ffab1a3584 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:57.490689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:57.492042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.492298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.492578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.493386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:57.498716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:57.506138 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1325/2674] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:57.510306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:57.520429 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129186e-cd6f-45cb-ab8c-0c461f1d1342 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:57.525080 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39027fa3-17a9-436e-a54b-3d1845301d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:57.525733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39027fa3-17a9-436e-a54b-3d1845301d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.526019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39027fa3-17a9-436e-a54b-3d1845301d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.526295 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39027fa3-17a9-436e-a54b-3d1845301d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.527209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39027fa3-17a9-436e-a54b-3d1845301d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:57.528630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39027fa3-17a9-436e-a54b-3d1845301d33 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:57.539083 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1350/2675] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:57.543328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:57.553267 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f55d-b561-4e7b-9ade-6353004cf7e6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:57.558518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:57.559394 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.559683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.559966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] PATH_INFO: `/v3/projects/2f7446686e1e4a3faab12463033a3d4e/users/adbd13857e404bfcbf11e80dd12df9df/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.575981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7446686e1e4a3faab12463033a3d4e, user_id=adbd13857e404bfcbf11e80dd12df9df, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:57.578850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:57.665671 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1326/2676] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:57 2022] PUT /identity/v3/projects/2f7446686e1e4a3faab12463033a3d4e/users/adbd13857e404bfcbf11e80dd12df9df/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:57.677439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd514-f118-4513-a02e-99a9923994c4 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.677738 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd514-f118-4513-a02e-99a9923994c4 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.678028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd514-f118-4513-a02e-99a9923994c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.705463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-38cbd514-f118-4513-a02e-99a9923994c4 None None] MFA Rules not processed for user `adbd13857e404bfcbf11e80dd12df9df`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:57.706707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-38cbd514-f118-4513-a02e-99a9923994c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:57.773922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1351/2677] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:57.778444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:57.832760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:57.844826 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664f8a8-f3e5-4998-9d6e-dc00062ed10f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:57.849546 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:57.850203 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:57.850503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:57.851347 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:57.852202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:57.854111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:58.040455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1327/2678] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:54:57 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:58.046658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd514-f118-4513-a02e-99a9923994c4 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:58.057906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd514-f118-4513-a02e-99a9923994c4 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:58.125850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:58.127170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:58.127628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:58.128070 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:58.133390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:58.133390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:58.213574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1352/2679] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:54:58 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:54:58.217625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:58.228520 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49728-d70c-4d7c-980b-0b0555ae9722 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:58.238859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:58.238859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:58.238859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:58.238859 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:58.239491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:58.239858 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:58.252086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1328/2680] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:54:58.256875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:58.263407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:58.267293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010182-1baf-46b4-b6d6-988cae87768a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:58.271970 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:58.272900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:58.273189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:58.273469 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] PATH_INFO: `/v3/projects/5b9c0527999249588da89b347fcdba68/users/23f75bc885154f50a94277c5f6a9f17e/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:58.292334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9c0527999249588da89b347fcdba68, user_id=23f75bc885154f50a94277c5f6a9f17e, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:58.295105 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:58.403877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1353/2681] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:58 2022] PUT /identity/v3/projects/5b9c0527999249588da89b347fcdba68/users/23f75bc885154f50a94277c5f6a9f17e/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:58.418117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:58.428705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955a71-f560-485c-9802-fc0ad922cccb admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:58.507582 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72652a1b-5026-4616-ba00-0730d89cc632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:58.508263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72652a1b-5026-4616-ba00-0730d89cc632 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:58.508551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72652a1b-5026-4616-ba00-0730d89cc632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:58.508864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72652a1b-5026-4616-ba00-0730d89cc632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:58.509601 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72652a1b-5026-4616-ba00-0730d89cc632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:58.511222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72652a1b-5026-4616-ba00-0730d89cc632 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:58.522145 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1329/2682] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:54:58.526992 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:58.537190 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8f23c-ab21-4c15-acaa-a7f43d72bb6c admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:58.542420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:58.543349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:58.543628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:58.543901 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] PATH_INFO: `/v3/projects/5b9c0527999249588da89b347fcdba68/users/23f75bc885154f50a94277c5f6a9f17e/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:58.545914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9c0527999249588da89b347fcdba68, user_id=23f75bc885154f50a94277c5f6a9f17e, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:58.548555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:58.558879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1354/2683] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:54:58 2022] PUT /identity/v3/projects/5b9c0527999249588da89b347fcdba68/users/23f75bc885154f50a94277c5f6a9f17e/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:54:58.569222 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1af5c-1ca4-48bc-8274-953ff094f6ca None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:58.572556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1af5c-1ca4-48bc-8274-953ff094f6ca None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:58.572556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1af5c-1ca4-48bc-8274-953ff094f6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:58.596745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7af1af5c-1ca4-48bc-8274-953ff094f6ca None None] MFA Rules not processed for user `23f75bc885154f50a94277c5f6a9f17e`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:54:58.601448 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7af1af5c-1ca4-48bc-8274-953ff094f6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:58.674963 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1330/2684] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:54:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:54:58.686670 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:58.688409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:58.754048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6575d28-f184-49e6-90fa-29770577b201 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:58.758558 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:58.759226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:58.759646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:58.759957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:58.775843 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:58.777357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:58.787137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1355/2685] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:59.151094 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1af5c-1ca4-48bc-8274-953ff094f6ca None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:59.163064 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1af5c-1ca4-48bc-8274-953ff094f6ca None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:59.166753 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9c013-faf8-4be5-bcae-a96541911c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:59.167409 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9c013-faf8-4be5-bcae-a96541911c07 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:59.167698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9c013-faf8-4be5-bcae-a96541911c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:59.167979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9c013-faf8-4be5-bcae-a96541911c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:59.219772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:54:59.236761 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9c013-faf8-4be5-bcae-a96541911c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:59.238521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9c013-faf8-4be5-bcae-a96541911c07 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:54:59.249504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1331/2686] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:54:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:54:59.886912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:54:59.888525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:54:59.978502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c96911-9b77-4daa-9756-6d51a331ed5e service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:54:59.983610 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:54:59.984883 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:54:59.985328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:54:59.985875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] PATH_INFO: `/v3/users/64e638d550a7429c8e91b64513a47c20` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:54:59.987468 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e638d550a7429c8e91b64513a47c20)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:54:59.989881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:00.216085 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e638d550a7429c8e91b64513a47c20'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:00.228998 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e638d550a7429c8e91b64513a47c20'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:00.229841 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e638d550a7429c8e91b64513a47c20'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:00.235336 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:00.237485 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e638d550a7429c8e91b64513a47c20'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:00.238150 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e638d550a7429c8e91b64513a47c20'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:00.344514 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1356/2687] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:54:59 2022] DELETE /identity/v3/users/64e638d550a7429c8e91b64513a47c20 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:00.348992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:00.352376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:00.352376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:00.367045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:00.368276 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:00.428459 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1332/2688] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:00.439055 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:00.440455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:00.498009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e318f3e-105d-4ef4-97aa-fc03496746bd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:00.501876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:00.502527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:00.502912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:00.503090 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:00.512484 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:00.513957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:00.524312 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1357/2689] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:00.986152 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:00.987583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:01.000200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6cd60-1109-4f2d-83df-96ebebc00251 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:01.007777 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:01.008694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:01.008990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:01.009272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] PATH_INFO: `/v3/projects/24d6f3d46ba74bf087e4b75741de86d8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:01.010507 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d6f3d46ba74bf087e4b75741de86d8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:01.012527 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:01.146474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d6f3d46ba74bf087e4b75741de86d8', 'request_id': 'req-1756195d-6356-4780-91b5-e4b48cc30171'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.153286 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d6f3d46ba74bf087e4b75741de86d8', 'request_id': 'req-1756195d-6356-4780-91b5-e4b48cc30171'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.207556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d6f3d46ba74bf087e4b75741de86d8', 'request_id': 'req-1756195d-6356-4780-91b5-e4b48cc30171'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.220166 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] The token cache is being invalidate because project 24d6f3d46ba74bf087e4b75741de86d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:01.220514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.221198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.222744 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1333/2690] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:00 2022] DELETE /identity/v3/projects/24d6f3d46ba74bf087e4b75741de86d8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:01.226900 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:01.240592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41fbf-d74f-404d-91db-4ee9223f5e1a service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:01.298413 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:01.299401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:01.299681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:01.299958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] PATH_INFO: `/v3/users/7e01c4a1795c4f4980d4cf12a598c9c1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:01.301174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e01c4a1795c4f4980d4cf12a598c9c1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:01.303256 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:01.470567 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e01c4a1795c4f4980d4cf12a598c9c1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.483935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e01c4a1795c4f4980d4cf12a598c9c1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.484866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e01c4a1795c4f4980d4cf12a598c9c1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.493666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e01c4a1795c4f4980d4cf12a598c9c1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.494577 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b8270fa-0761-4f89-91f8-f0408da804f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e01c4a1795c4f4980d4cf12a598c9c1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.579473 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1358/2691] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:01 2022] DELETE /identity/v3/users/7e01c4a1795c4f4980d4cf12a598c9c1 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:01.612294 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:01.613990 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:01.629632 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756195d-6356-4780-91b5-e4b48cc30171 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:01.709369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:01.710849 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:01.711143 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:01.711431 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] PATH_INFO: `/v3/users/046d0f5ade9e41e7bfd913b08d933166` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:01.712706 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=046d0f5ade9e41e7bfd913b08d933166)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:01.714720 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:01.727606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:01.919393 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '046d0f5ade9e41e7bfd913b08d933166'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.926436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '046d0f5ade9e41e7bfd913b08d933166'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.984403 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '046d0f5ade9e41e7bfd913b08d933166'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.985098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '046d0f5ade9e41e7bfd913b08d933166'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:01.985709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '046d0f5ade9e41e7bfd913b08d933166'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.003570 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1334/2692] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:01 2022] DELETE /identity/v3/users/046d0f5ade9e41e7bfd913b08d933166 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:02.008679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.008985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.009268 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.028453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:02.029715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:02.034766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:02.066387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.067836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:02.098703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1359/2693] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:02.102539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.103980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:02.144102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e287bc-3f90-43fb-a350-464b929e9c90 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:02.150953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:02.157395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.157395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.157395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] PATH_INFO: `/v3/projects/952700b80a394e94a0400330e994c8f3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.157395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=952700b80a394e94a0400330e994c8f3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:02.157395 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:02.168808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cc6d1-5c28-4c41-86d1-2b27ed767f0c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:02.180363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d90ab-09f9-4c21-b628-7b7b630cbc59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:02.181193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d90ab-09f9-4c21-b628-7b7b630cbc59 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.183771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d90ab-09f9-4c21-b628-7b7b630cbc59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.183771 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d90ab-09f9-4c21-b628-7b7b630cbc59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.191875 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d90ab-09f9-4c21-b628-7b7b630cbc59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:02.193350 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d90ab-09f9-4c21-b628-7b7b630cbc59 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:02.203441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1360/2694] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:02.294454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952700b80a394e94a0400330e994c8f3', 'request_id': 'req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.300716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952700b80a394e94a0400330e994c8f3', 'request_id': 'req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.413755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952700b80a394e94a0400330e994c8f3', 'request_id': 'req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.424835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] The token cache is being invalidate because project 952700b80a394e94a0400330e994c8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:02.425126 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.425843 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-18c7b1fa-fb6f-4deb-aa68-d2b3b81537d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.427656 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1335/2695] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:02 2022] DELETE /identity/v3/projects/952700b80a394e94a0400330e994c8f3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:02.434986 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667be4a-ab7e-43df-8d81-2411e2fbf654 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.436051 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667be4a-ab7e-43df-8d81-2411e2fbf654 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.436401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667be4a-ab7e-43df-8d81-2411e2fbf654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.460339 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a667be4a-ab7e-43df-8d81-2411e2fbf654 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:02.461750 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a667be4a-ab7e-43df-8d81-2411e2fbf654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:02.525334 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1361/2696] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:02.533078 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6950e-a033-4e00-bcf1-6c86b19f2785 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.533481 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6950e-a033-4e00-bcf1-6c86b19f2785 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.533829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6950e-a033-4e00-bcf1-6c86b19f2785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.552989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-4aa6950e-a033-4e00-bcf1-6c86b19f2785 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:02.554943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-4aa6950e-a033-4e00-bcf1-6c86b19f2785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:02.563018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1336/2697] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:02.572253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667be4a-ab7e-43df-8d81-2411e2fbf654 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.582189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667be4a-ab7e-43df-8d81-2411e2fbf654 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:02.586908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:02.587563 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.587850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.588129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.588896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:55:02.589208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:02.590927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:02.603742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1362/2698] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:55:02.608029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6950e-a033-4e00-bcf1-6c86b19f2785 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.618478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6950e-a033-4e00-bcf1-6c86b19f2785 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:02.623140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:02.624042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.624325 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.624615 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.625362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:02.627338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:02.637454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:02.655042 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.656803 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:02.673159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb170fc9-f1f2-4d92-a96e-272d0e627d20 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:02.678202 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:02.679806 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.680161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.680546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] PATH_INFO: `/v3/projects/1e37964d56b545a991b28be9a4f997f1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.682050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e37964d56b545a991b28be9a4f997f1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:02.684317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:02.723065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1337/2699] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:02 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:02.726341 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.768974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:02.781293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e37964d56b545a991b28be9a4f997f1', 'request_id': 'req-6283b781-2d13-449c-a0eb-44e9879d8ead'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.789735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e37964d56b545a991b28be9a4f997f1', 'request_id': 'req-6283b781-2d13-449c-a0eb-44e9879d8ead'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.795288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e37964d56b545a991b28be9a4f997f1', 'request_id': 'req-6283b781-2d13-449c-a0eb-44e9879d8ead'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.809610 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc787ab-a07b-48d7-b297-501b42406f52 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:02.865786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:02.871359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.871359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.871359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.871359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:02.871359 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:02.892703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] The token cache is being invalidate because project 1e37964d56b545a991b28be9a4f997f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:02.893001 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.893716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:02.895401 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1363/2700] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:02 2022] DELETE /identity/v3/projects/1e37964d56b545a991b28be9a4f997f1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:02.899160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.901363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:02.917064 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283b781-2d13-449c-a0eb-44e9879d8ead admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:02.921799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:02.922676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:02.922961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:02.923240 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] PATH_INFO: `/v3/users/c39001c188ac4acda8557b797f2e9403` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:02.924488 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39001c188ac4acda8557b797f2e9403)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:02.926552 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:02.984543 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1338/2701] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:02 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:55:02.988808 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:02.990404 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:03.002181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef99fb6-2463-4238-b93b-e0193e5324cd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.007066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.007725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.009303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.009303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.009303 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.010640 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.021114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1339/2702] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:55:03.025322 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.035116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408edf58-3e7d-49cf-8ca5-27494fed11d6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.041297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.042278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.042711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.043195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.044336 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.047210 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.063916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1340/2703] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:55:03.070091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.082606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9f87-a54a-4edb-9466-7682b3efdad3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.088356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b306c-8018-4711-a81a-50e7b716afb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.089298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b306c-8018-4711-a81a-50e7b716afb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.089569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b306c-8018-4711-a81a-50e7b716afb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.089851 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b306c-8018-4711-a81a-50e7b716afb7 admin admin] PATH_INFO: `/v3/projects/eada0887db344253af749745eb6f5cff/users/6e1ed73f0c6147d6a83e6f5e6d8314f2/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.112388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b306c-8018-4711-a81a-50e7b716afb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eada0887db344253af749745eb6f5cff, user_id=6e1ed73f0c6147d6a83e6f5e6d8314f2, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.116983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b306c-8018-4711-a81a-50e7b716afb7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.153940 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39001c188ac4acda8557b797f2e9403'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:03.167731 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39001c188ac4acda8557b797f2e9403'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:03.168497 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39001c188ac4acda8557b797f2e9403'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:03.175921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39001c188ac4acda8557b797f2e9403'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:03.176746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39001c188ac4acda8557b797f2e9403'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:03.228148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1341/2704] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:03 2022] PUT /identity/v3/projects/eada0887db344253af749745eb6f5cff/users/6e1ed73f0c6147d6a83e6f5e6d8314f2/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:03.240987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdf214-ad12-45f8-a010-b328e6210c17 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.241282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdf214-ad12-45f8-a010-b328e6210c17 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.241693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdf214-ad12-45f8-a010-b328e6210c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.259217 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1364/2705] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:02 2022] DELETE /identity/v3/users/c39001c188ac4acda8557b797f2e9403 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:03.269195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bbfdf214-ad12-45f8-a010-b328e6210c17 None None] MFA Rules not processed for user `6e1ed73f0c6147d6a83e6f5e6d8314f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:03.270412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbfdf214-ad12-45f8-a010-b328e6210c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:03.309997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:03.380639 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1342/2706] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:03.384302 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.386043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:03.464086 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555c766-a5a6-4c19-8f28-4575f9d3f0eb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.468758 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.469420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.469702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.470178 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.470763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.472700 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.597082 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1365/2707] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:03 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:03.601031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdf214-ad12-45f8-a010-b328e6210c17 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.612515 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdf214-ad12-45f8-a010-b328e6210c17 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.679241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.679984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.680188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.680473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.681202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.683158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.725867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.725867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:03.744300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b10be-a795-4e2b-83fb-05bf5fe23d29 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.749716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.750594 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.750880 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.751161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] PATH_INFO: `/v3/projects/7e3635d49c9b4c619156e1b84ce60ebf` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.753501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3635d49c9b4c619156e1b84ce60ebf)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.754453 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.775745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:03.796211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1343/2708] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:03 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:55:03.800110 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.810417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd8cbe-33c7-44dc-81b3-188486e1ca72 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.817019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.817681 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.817968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.818249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.818974 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.821219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.832941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1344/2709] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:55:03.837144 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.846998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a13450-01e0-4f4f-85c8-d082f94b40d4 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:03.851890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:03.852838 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:03.853136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:03.853416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] PATH_INFO: `/v3/projects/39b3d722dd1f41bf871ac0f951137054/users/688cc4543d1040a692f55c3e064cdf17/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:03.869383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b3d722dd1f41bf871ac0f951137054, user_id=688cc4543d1040a692f55c3e064cdf17, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:03.872277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:03.950799 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1345/2710] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:03 2022] PUT /identity/v3/projects/39b3d722dd1f41bf871ac0f951137054/users/688cc4543d1040a692f55c3e064cdf17/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:03.954779 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:03.960403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3635d49c9b4c619156e1b84ce60ebf', 'request_id': 'req-86c21896-6856-4835-a01b-39841a63e748'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:03.967863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3635d49c9b4c619156e1b84ce60ebf', 'request_id': 'req-86c21896-6856-4835-a01b-39841a63e748'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:03.973629 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3635d49c9b4c619156e1b84ce60ebf', 'request_id': 'req-86c21896-6856-4835-a01b-39841a63e748'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:04.016368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0afe2-47fd-43f1-9f1a-8f94f9773313 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.049921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:04.060559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] The token cache is being invalidate because project 7e3635d49c9b4c619156e1b84ce60ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:04.060959 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:04.061726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-86c21896-6856-4835-a01b-39841a63e748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:04.064096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1366/2711] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:03 2022] DELETE /identity/v3/projects/7e3635d49c9b4c619156e1b84ce60ebf => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:04.069972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72037827-d8ed-4136-b7f4-6967cb05aeb7 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.070342 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72037827-d8ed-4136-b7f4-6967cb05aeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.070690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72037827-d8ed-4136-b7f4-6967cb05aeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.086135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.086875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.087093 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.087370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.088101 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.089773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.090949 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-72037827-d8ed-4136-b7f4-6967cb05aeb7 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:04.092941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-72037827-d8ed-4136-b7f4-6967cb05aeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:04.101669 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1346/2712] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:55:04.105227 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1367/2713] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:04.107198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.108930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:04.117961 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0ff6a-17ed-4c39-ab9c-aebc91da7cae None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.118253 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0ff6a-17ed-4c39-ab9c-aebc91da7cae None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.118539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0ff6a-17ed-4c39-ab9c-aebc91da7cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.127567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48773c8-037d-41e4-b054-ad1aed5376e6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.132920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455930b7-ddde-4f57-98a3-e4ea2f016a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.133802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455930b7-ddde-4f57-98a3-e4ea2f016a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.134081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455930b7-ddde-4f57-98a3-e4ea2f016a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.134353 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455930b7-ddde-4f57-98a3-e4ea2f016a17 admin admin] PATH_INFO: `/v3/projects/39b3d722dd1f41bf871ac0f951137054/users/688cc4543d1040a692f55c3e064cdf17/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.136419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455930b7-ddde-4f57-98a3-e4ea2f016a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b3d722dd1f41bf871ac0f951137054, user_id=688cc4543d1040a692f55c3e064cdf17, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.136912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a5e0ff6a-17ed-4c39-ab9c-aebc91da7cae None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:04.138126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a5e0ff6a-17ed-4c39-ab9c-aebc91da7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:04.139174 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455930b7-ddde-4f57-98a3-e4ea2f016a17 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.146144 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1368/2714] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:04.150118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1347/2715] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:04 2022] PUT /identity/v3/projects/39b3d722dd1f41bf871ac0f951137054/users/688cc4543d1040a692f55c3e064cdf17/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:04.156346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0ff6a-17ed-4c39-ab9c-aebc91da7cae None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.159418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0ff6a-17ed-4c39-ab9c-aebc91da7cae None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.162039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.162039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.162039 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.190665 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] MFA Rules not processed for user `688cc4543d1040a692f55c3e064cdf17`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:04.192043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:04.227128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.227714 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.227997 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.228272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.229030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:55:04.229335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.230937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.252216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1369/2716] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:55:04.256364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.269917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7b12f-ab6c-4f7e-a309-74bbc0b54023 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.275651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.276797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.277829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.278136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.279242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.281507 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.284909 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1348/2717] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:04.299575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.301057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:04.357534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d84977-dc84-4666-a41e-668a4d81f863 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.361587 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.362472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.362472 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.362783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.378419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1370/2718] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:04 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:55:04.378536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.378536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.381170 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.387217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1349/2719] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:04.446636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83911ee3-536a-4534-9f95-774331dde0ca admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.451662 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.452273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.452579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.452868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.453604 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.455565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.469655 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:04.522908 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1371/2720] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:04 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:04.526919 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.537715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aaa74-c434-40cc-9079-eb77c4d99458 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.542818 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.543556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.543828 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.544106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.544847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.547206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.563180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1350/2721] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:04.565809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.576176 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dabaa-20bc-4e6a-80cd-68c45b7b7db3 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.581479 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7cfa7-4d2a-47f2-b35c-d6c4c56ca18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.582158 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7cfa7-4d2a-47f2-b35c-d6c4c56ca18e admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.589358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7cfa7-4d2a-47f2-b35c-d6c4c56ca18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.589358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7cfa7-4d2a-47f2-b35c-d6c4c56ca18e admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.589358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7cfa7-4d2a-47f2-b35c-d6c4c56ca18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.589358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7cfa7-4d2a-47f2-b35c-d6c4c56ca18e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.601301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1372/2722] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:04.602291 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.615966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a949234-3dc4-4c88-b702-ef2ffe4fb6de admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.620918 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.621812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.622100 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.622384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] PATH_INFO: `/v3/projects/632afe8a74304c8cb5d65135bd212ed5/users/f5234efe1b154f5bb064cf3996a96a31/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.638523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=632afe8a74304c8cb5d65135bd212ed5, user_id=f5234efe1b154f5bb064cf3996a96a31, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.641263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:04.762165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1351/2723] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:04 2022] PUT /identity/v3/projects/632afe8a74304c8cb5d65135bd212ed5/users/f5234efe1b154f5bb064cf3996a96a31/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:04.774036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694d08b-cdf8-4fd9-9a96-8a34ff6c99a8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.774259 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694d08b-cdf8-4fd9-9a96-8a34ff6c99a8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.774542 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694d08b-cdf8-4fd9-9a96-8a34ff6c99a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.805919 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2694d08b-cdf8-4fd9-9a96-8a34ff6c99a8 None None] MFA Rules not processed for user `f5234efe1b154f5bb064cf3996a96a31`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:04.807133 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2694d08b-cdf8-4fd9-9a96-8a34ff6c99a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:04.877415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1373/2724] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:04.882497 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:04.892862 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:04.943812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9c650-5bdc-44df-a17c-d6a8aea93935 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:04.949102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:04.949755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:04.950046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:04.950350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:04.951015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:04.953035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:05.090312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1352/2725] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:04 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:05.094004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694d08b-cdf8-4fd9-9a96-8a34ff6c99a8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:05.105833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694d08b-cdf8-4fd9-9a96-8a34ff6c99a8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:05.166278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:05.174417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.174417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.174417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.174417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:05.174417 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:05.254226 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1374/2726] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:05 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:55:05.259173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:05.270099 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791bd6f-dbd7-4407-bf5b-acc3a9ea90a5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:05.274895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:05.275564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.275853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.276135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.276897 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:05.278569 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:05.290523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1353/2727] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:05.296716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:05.308060 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13802f5b-f9d9-4d25-91c0-35eeeec5784a admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:05.312907 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:05.313797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.314128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.314410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] PATH_INFO: `/v3/projects/dac767e74d3540d4a407fc9e12e18be4/users/c6a99144bfbd4812a92bf2a251364383/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.333254 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac767e74d3540d4a407fc9e12e18be4, user_id=c6a99144bfbd4812a92bf2a251364383, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:05.335902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:05.401837 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1375/2728] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:05 2022] PUT /identity/v3/projects/dac767e74d3540d4a407fc9e12e18be4/users/c6a99144bfbd4812a92bf2a251364383/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:05.405899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:05.416004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38175d9a-a7ed-423e-8cf0-28052d26f9b0 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:05.480365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de623f-c7fc-417d-9cf4-a41d142034f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:05.481185 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de623f-c7fc-417d-9cf4-a41d142034f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.481484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de623f-c7fc-417d-9cf4-a41d142034f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.481756 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de623f-c7fc-417d-9cf4-a41d142034f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.482657 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de623f-c7fc-417d-9cf4-a41d142034f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:05.484372 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de623f-c7fc-417d-9cf4-a41d142034f6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:05.498523 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1354/2729] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:05.499892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:05.511262 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6ae36-01ff-499c-8846-dbb6ade9b401 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:05.515918 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:05.516839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.517121 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.517393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] PATH_INFO: `/v3/projects/dac767e74d3540d4a407fc9e12e18be4/users/c6a99144bfbd4812a92bf2a251364383/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.519412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac767e74d3540d4a407fc9e12e18be4, user_id=c6a99144bfbd4812a92bf2a251364383, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:05.522211 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:05.532590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1376/2730] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:05 2022] PUT /identity/v3/projects/dac767e74d3540d4a407fc9e12e18be4/users/c6a99144bfbd4812a92bf2a251364383/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:05.544264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6953-ad22-4032-b70e-6d8b1b0153ec None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.544529 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6953-ad22-4032-b70e-6d8b1b0153ec None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.544868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6953-ad22-4032-b70e-6d8b1b0153ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.571995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-879a6953-ad22-4032-b70e-6d8b1b0153ec None None] MFA Rules not processed for user `c6a99144bfbd4812a92bf2a251364383`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:05.573255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-879a6953-ad22-4032-b70e-6d8b1b0153ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:05.641969 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1355/2731] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:05.668177 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:05.697356 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:05.726787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4773-896e-4172-9eac-33ff7d4f5a0b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:05.730794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:05.731448 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.731780 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.732079 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.789472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:05.790927 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:05.806872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1377/2732] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:05.815912 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a6953-ad22-4032-b70e-6d8b1b0153ec None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:05.823604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:05.890255 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a6953-ad22-4032-b70e-6d8b1b0153ec None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:05.895916 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:05.896832 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:05.897358 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:05.897755 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] PATH_INFO: `/v3/users/6e1ed73f0c6147d6a83e6f5e6d8314f2` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:05.898984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1ed73f0c6147d6a83e6f5e6d8314f2)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:05.900972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:06.091232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1ed73f0c6147d6a83e6f5e6d8314f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.102051 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1ed73f0c6147d6a83e6f5e6d8314f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.144666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:06.146097 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:06.161175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c55de-0eff-4359-83d0-fc6bf70f950e service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:06.166575 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:06.167493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:06.168335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:06.168650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] PATH_INFO: `/v3/users/adbd13857e404bfcbf11e80dd12df9df` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:06.170094 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adbd13857e404bfcbf11e80dd12df9df)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:06.172189 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:06.173724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1ed73f0c6147d6a83e6f5e6d8314f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.174775 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1ed73f0c6147d6a83e6f5e6d8314f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.175410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-79e71a5c-d7b5-491b-8d06-8a74402bc4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1ed73f0c6147d6a83e6f5e6d8314f2'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.198556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1356/2733] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:05 2022] DELETE /identity/v3/users/6e1ed73f0c6147d6a83e6f5e6d8314f2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:06.203469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:06.203771 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:06.204111 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:06.223491 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:06.224785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:06.283831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1357/2734] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:06.294077 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:06.295578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:06.323241 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:06.352277 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f11bf-e90b-4e29-ab4a-a3312394e734 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:06.356067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567b41a-1516-4545-910f-28ddecd1dd95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:06.356817 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b41a-1516-4545-910f-28ddecd1dd95 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:06.357134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b41a-1516-4545-910f-28ddecd1dd95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:06.357445 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b41a-1516-4545-910f-28ddecd1dd95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:06.367365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567b41a-1516-4545-910f-28ddecd1dd95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:06.368937 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567b41a-1516-4545-910f-28ddecd1dd95 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:06.380578 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1358/2735] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:06.413606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adbd13857e404bfcbf11e80dd12df9df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.447317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adbd13857e404bfcbf11e80dd12df9df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.448773 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adbd13857e404bfcbf11e80dd12df9df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.459129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adbd13857e404bfcbf11e80dd12df9df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.463557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adbd13857e404bfcbf11e80dd12df9df'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:06.566283 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1378/2736] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:06 2022] DELETE /identity/v3/users/adbd13857e404bfcbf11e80dd12df9df => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:06.570821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2fe14-3ddf-4707-83dd-3c5fc076d31a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:06.571356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2fe14-3ddf-4707-83dd-3c5fc076d31a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:06.571683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2fe14-3ddf-4707-83dd-3c5fc076d31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:06.595046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-05c2fe14-3ddf-4707-83dd-3c5fc076d31a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:06.595046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-05c2fe14-3ddf-4707-83dd-3c5fc076d31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:06.661492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1359/2737] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:06.675364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:06.677658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:06.747583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a67f82-fa04-43b4-a3c3-a459834fd715 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:06.752208 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:06.752917 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:06.753199 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:06.753477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:06.761878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:06.765098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:06.766681 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:06.775278 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2fe14-3ddf-4707-83dd-3c5fc076d31a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:06.779105 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1379/2738] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:06.789876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2fe14-3ddf-4707-83dd-3c5fc076d31a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:06.793661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:06.794317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:06.794611 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:06.795023 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:06.796929 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:06.858198 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:06.860045 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:06.867510 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:06.869200 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:06.881339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1360/2739] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:06.883159 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1481a-9133-47cc-a029-588a133d8220 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:06.886897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:06.887779 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:06.888065 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:06.888346 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] PATH_INFO: `/v3/projects/eada0887db344253af749745eb6f5cff` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:06.889794 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eada0887db344253af749745eb6f5cff)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:06.891770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:07.019184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eada0887db344253af749745eb6f5cff', 'request_id': 'req-8713f3b2-e5fa-4b34-afa3-3bac6912880f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.025688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eada0887db344253af749745eb6f5cff', 'request_id': 'req-8713f3b2-e5fa-4b34-afa3-3bac6912880f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.033281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eada0887db344253af749745eb6f5cff', 'request_id': 'req-8713f3b2-e5fa-4b34-afa3-3bac6912880f'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.153054 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] The token cache is being invalidate because project eada0887db344253af749745eb6f5cff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:07.153565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.154397 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.156369 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1380/2740] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:06 2022] DELETE /identity/v3/projects/eada0887db344253af749745eb6f5cff => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:07.160130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:07.171580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebceea-b493-4b4e-8176-71f55a8ec42a service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:07.232410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:07.233494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:07.233783 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:07.234065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] PATH_INFO: `/v3/users/688cc4543d1040a692f55c3e064cdf17` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:07.235302 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=688cc4543d1040a692f55c3e064cdf17)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:07.237387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:07.240183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=100 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:07.340805 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:07.342835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:07.359624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f3b2-e5fa-4b34-afa3-3bac6912880f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:07.375957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:07.376945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:07.377235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:07.377532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] PATH_INFO: `/v3/projects/2f7446686e1e4a3faab12463033a3d4e` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:07.378846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7446686e1e4a3faab12463033a3d4e)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:07.380795 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:07.468805 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688cc4543d1040a692f55c3e064cdf17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.478007 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688cc4543d1040a692f55c3e064cdf17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.504999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7446686e1e4a3faab12463033a3d4e', 'request_id': 'req-dd043fb9-ac61-4bab-bc61-b38ed5a31113'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.511937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7446686e1e4a3faab12463033a3d4e', 'request_id': 'req-dd043fb9-ac61-4bab-bc61-b38ed5a31113'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.518682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7446686e1e4a3faab12463033a3d4e', 'request_id': 'req-dd043fb9-ac61-4bab-bc61-b38ed5a31113'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.561996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688cc4543d1040a692f55c3e064cdf17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.562688 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688cc4543d1040a692f55c3e064cdf17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.563370 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688cc4543d1040a692f55c3e064cdf17'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.587576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1361/2741] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:07 2022] DELETE /identity/v3/users/688cc4543d1040a692f55c3e064cdf17 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:07.610852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] The token cache is being invalidate because project 2f7446686e1e4a3faab12463033a3d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:07.611427 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.612236 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.617316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1381/2742] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:07 2022] DELETE /identity/v3/projects/2f7446686e1e4a3faab12463033a3d4e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:07.618443 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:07.619879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:07.689498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604263f2-5b01-464e-9bed-e85a6bc26b9a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:07.694180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:07.695098 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:07.695383 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:07.695658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] PATH_INFO: `/v3/users/23f75bc885154f50a94277c5f6a9f17e` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:07.696986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f75bc885154f50a94277c5f6a9f17e)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:07.699384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:07.893879 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f75bc885154f50a94277c5f6a9f17e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.903887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f75bc885154f50a94277c5f6a9f17e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.968551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f75bc885154f50a94277c5f6a9f17e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.969312 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f75bc885154f50a94277c5f6a9f17e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.970116 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-106f2526-f27e-42f5-87dc-8fd52ebaf9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f75bc885154f50a94277c5f6a9f17e'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:07.993880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:07.998959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1362/2743] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:07 2022] DELETE /identity/v3/users/23f75bc885154f50a94277c5f6a9f17e => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:08.056990 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.058449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:08.155693 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043fb9-ac61-4bab-bc61-b38ed5a31113 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.159444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:08.221022 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.222161 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.222476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.222828 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] PATH_INFO: `/v3/projects/39b3d722dd1f41bf871ac0f951137054` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.224168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b3d722dd1f41bf871ac0f951137054)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.230066 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.351381 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b3d722dd1f41bf871ac0f951137054', 'request_id': 'req-70d31870-ce3b-443d-b35b-032e2d51da54'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.358433 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b3d722dd1f41bf871ac0f951137054', 'request_id': 'req-70d31870-ce3b-443d-b35b-032e2d51da54'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.364322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b3d722dd1f41bf871ac0f951137054', 'request_id': 'req-70d31870-ce3b-443d-b35b-032e2d51da54'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.475815 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] The token cache is being invalidate because project 39b3d722dd1f41bf871ac0f951137054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:08.476040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.476835 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.478517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1382/2744] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:08 2022] DELETE /identity/v3/projects/39b3d722dd1f41bf871ac0f951137054 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:08.483532 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e949d-53a0-4e59-b4e5-39990ea9aff6 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.483831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e949d-53a0-4e59-b4e5-39990ea9aff6 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.484140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e949d-53a0-4e59-b4e5-39990ea9aff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.504575 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-de2e949d-53a0-4e59-b4e5-39990ea9aff6 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:08.505847 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-de2e949d-53a0-4e59-b4e5-39990ea9aff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:08.509865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.512036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:08.578106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1363/2745] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:08.585759 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2307cb-7fb3-4c79-a8b9-6c57d269db9f None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.586063 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2307cb-7fb3-4c79-a8b9-6c57d269db9f None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.586342 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2307cb-7fb3-4c79-a8b9-6c57d269db9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.591535 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d31870-ce3b-443d-b35b-032e2d51da54 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.596540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.597437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.597725 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.598005 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] PATH_INFO: `/v3/projects/5b9c0527999249588da89b347fcdba68` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.599221 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b9c0527999249588da89b347fcdba68)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.601826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.604456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-3b2307cb-7fb3-4c79-a8b9-6c57d269db9f None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:08.605689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-3b2307cb-7fb3-4c79-a8b9-6c57d269db9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:08.615604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1364/2746] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:08.620557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2307cb-7fb3-4c79-a8b9-6c57d269db9f None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.631095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2307cb-7fb3-4c79-a8b9-6c57d269db9f None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.636901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.637560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.637845 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.638269 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.639135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:55:08.643802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.643802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.655477 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1365/2747] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:55:08.659924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.671772 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba0a46-d2d6-4dce-8bd1-f05fe9c91294 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.676521 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.677462 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.677612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.678831 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.679538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.681542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.711441 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b9c0527999249588da89b347fcdba68', 'request_id': 'req-15e86fe4-6649-4ae7-87af-267f28850ee6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.717896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b9c0527999249588da89b347fcdba68', 'request_id': 'req-15e86fe4-6649-4ae7-87af-267f28850ee6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.724024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b9c0527999249588da89b347fcdba68', 'request_id': 'req-15e86fe4-6649-4ae7-87af-267f28850ee6'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.791128 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] The token cache is being invalidate because project 5b9c0527999249588da89b347fcdba68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:08.791419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.792168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-15e86fe4-6649-4ae7-87af-267f28850ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:08.793095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1366/2748] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:08 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:08.794294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1383/2749] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:08 2022] DELETE /identity/v3/projects/5b9c0527999249588da89b347fcdba68 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:08.797091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.798792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300a216-8fa3-4ba5-9f5d-e66a6a6b0a02 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.799098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300a216-8fa3-4ba5-9f5d-e66a6a6b0a02 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.799504 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300a216-8fa3-4ba5-9f5d-e66a6a6b0a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.799679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:08.819152 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-2300a216-8fa3-4ba5-9f5d-e66a6a6b0a02 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:08.820384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2300a216-8fa3-4ba5-9f5d-e66a6a6b0a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:08.881794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdee2b7-484b-42e0-9924-544ae1fb66f8 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.886329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1384/2750] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:08.886606 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.887350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.887641 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.887920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.888650 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.890693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.893644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dbfa1-c09e-4dd0-a75b-5707be85e045 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.894148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dbfa1-c09e-4dd0-a75b-5707be85e045 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.894616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dbfa1-c09e-4dd0-a75b-5707be85e045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.915388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-671dbfa1-c09e-4dd0-a75b-5707be85e045 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:08.916736 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-671dbfa1-c09e-4dd0-a75b-5707be85e045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:08.924942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1385/2751] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:08.929881 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dbfa1-c09e-4dd0-a75b-5707be85e045 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.938957 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dbfa1-c09e-4dd0-a75b-5707be85e045 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.943741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.944412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.944737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.945018 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.945735 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:55:08.946052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.947624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.951117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1367/2752] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:08 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:08.955280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.964538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1386/2753] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:55:08.965570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.966129 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6ba7e-55d9-45b3-bdfe-efd61da145bd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.970979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.971538 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.971829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.972115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.972881 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.974504 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.976836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3452b-7c52-4b4b-a8e0-4b636de79db4 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:08.981879 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:08.982536 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:08.982851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:08.983186 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:08.983863 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:08.985542 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1368/2754] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:08.986142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:08.989264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:08.999473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be092a2d-d18d-45c4-8db0-423fc864d690 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.007137 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.007972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.008264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.008794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.009531 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.011368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.026067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1369/2755] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:55:09.029195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:09.039857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93331869-ee05-4403-aae9-35069265e70b admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.044654 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ae90-dadf-4a50-a4ac-d09ad94975a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.045533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ae90-dadf-4a50-a4ac-d09ad94975a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.045910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ae90-dadf-4a50-a4ac-d09ad94975a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.046195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ae90-dadf-4a50-a4ac-d09ad94975a8 admin admin] PATH_INFO: `/v3/projects/4a434570640b49848a9c3020208efcab/users/f4cc8e2a600144fe9a0a65cb8f2c5b83/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.060809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ae90-dadf-4a50-a4ac-d09ad94975a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a434570640b49848a9c3020208efcab, user_id=f4cc8e2a600144fe9a0a65cb8f2c5b83, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.063559 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ae90-dadf-4a50-a4ac-d09ad94975a8 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.086763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1387/2756] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:08 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:09.088074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:09.128352 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1370/2757] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:09 2022] PUT /identity/v3/projects/4a434570640b49848a9c3020208efcab/users/f4cc8e2a600144fe9a0a65cb8f2c5b83/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:09.140729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.141139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.141553 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.159081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:09.168387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a2f48-81aa-4672-9b76-237d1d3912c8 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.180793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] MFA Rules not processed for user `f4cc8e2a600144fe9a0a65cb8f2c5b83`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:09.182412 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:09.240546 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.241877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.242921 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.243391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.244108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.246829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.282150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1371/2758] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:09.295616 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:09.297272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:09.355861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb4482-efe9-48c6-ad21-1b72b6a4690a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.359492 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.360187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.360525 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.361195 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.370509 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.372019 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.382456 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1372/2759] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:09.383434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:09.390470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1388/2760] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:09 2022] POST /identity/v3/users => generated 286 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:55:09.394357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:09.407433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113cc47-451d-4624-9642-e8e81a9a8b15 service cinder] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.413981 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.414704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.414995 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.416906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.416906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.418217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.433800 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1373/2761] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:09.436106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:09.452072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d02d3-3573-4200-b5d6-2ebae4bf7a01 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.459638 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48d626-4b5e-4f4f-9a59-e82c87062a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.460953 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d626-4b5e-4f4f-9a59-e82c87062a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.461390 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d626-4b5e-4f4f-9a59-e82c87062a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.461716 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d626-4b5e-4f4f-9a59-e82c87062a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.463303 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48d626-4b5e-4f4f-9a59-e82c87062a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.465601 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48d626-4b5e-4f4f-9a59-e82c87062a08 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.481423 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1389/2762] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:55:09.482868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:09.494250 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285eda6-34e3-4646-9f5c-2a2f136991c3 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.500566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.501580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.508763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.508763 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] PATH_INFO: `/v3/projects/d9a1495d03fa4a76866a8784e3281a10/users/62fbd947e2a14d11b5cfeb26730347e5/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.523603 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a1495d03fa4a76866a8784e3281a10, user_id=62fbd947e2a14d11b5cfeb26730347e5, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.526984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.620757 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1374/2763] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:09 2022] PUT /identity/v3/projects/d9a1495d03fa4a76866a8784e3281a10/users/62fbd947e2a14d11b5cfeb26730347e5/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:09.626093 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054d442-cf86-44b0-9931-cdd59ec607fb None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.630695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054d442-cf86-44b0-9931-cdd59ec607fb None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.631052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054d442-cf86-44b0-9931-cdd59ec607fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.650297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d054d442-cf86-44b0-9931-cdd59ec607fb None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:09.651515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d054d442-cf86-44b0-9931-cdd59ec607fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:09.721951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1390/2764] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:09.732620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:09.734466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:09.795944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceebd8-dc0f-44c1-b315-50c7acfa1f76 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:09.799700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:09.800307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:09.800591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:09.800895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:09.810382 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:09.811854 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:09.822591 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1375/2765] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:11.156703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:11.455745 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0582eb-4b3d-4b31-9b42-70bffdbeebf3 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:11.457142 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0582eb-4b3d-4b31-9b42-70bffdbeebf3 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:11.458363 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0582eb-4b3d-4b31-9b42-70bffdbeebf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:11.504932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-8a0582eb-4b3d-4b31-9b42-70bffdbeebf3 None None] MFA Rules not processed for user `62fbd947e2a14d11b5cfeb26730347e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:11.507108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8a0582eb-4b3d-4b31-9b42-70bffdbeebf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:11.610847 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1391/2766] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:11.625739 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:11.628046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:11.635295 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:11.695910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db086-5dbf-453f-b247-50d3db75ee45 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:11.699550 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbb308-4fc8-4299-8aab-0dee083d2df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:11.700201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb308-4fc8-4299-8aab-0dee083d2df9 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:11.700490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb308-4fc8-4299-8aab-0dee083d2df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:11.700869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb308-4fc8-4299-8aab-0dee083d2df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:11.709311 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbb308-4fc8-4299-8aab-0dee083d2df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:11.716584 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbb308-4fc8-4299-8aab-0dee083d2df9 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:11.722966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1376/2767] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:12.138407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0582eb-4b3d-4b31-9b42-70bffdbeebf3 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:12.148946 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0582eb-4b3d-4b31-9b42-70bffdbeebf3 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:12.152432 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:12.153107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:12.153394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:12.153673 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:12.155480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:12.227724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:12.234246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:12.249889 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1392/2768] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:12.882332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:13.602329 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fd501-1c15-49a9-8c26-ef629d2a6ecb None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:13.602842 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fd501-1c15-49a9-8c26-ef629d2a6ecb None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:13.602926 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fd501-1c15-49a9-8c26-ef629d2a6ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:13.620577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-439fd501-1c15-49a9-8c26-ef629d2a6ecb None None] MFA Rules not processed for user `62fbd947e2a14d11b5cfeb26730347e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:13.621899 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-439fd501-1c15-49a9-8c26-ef629d2a6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:13.629363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1377/2769] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:17.350688 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:17.352104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:17.443616 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902689-56b7-41d5-aaef-d86b93fccab3 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:17.448726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc67bc9-285f-4dad-b876-3f891c52ab26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:17.449373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67bc9-285f-4dad-b876-3f891c52ab26 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:17.449661 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67bc9-285f-4dad-b876-3f891c52ab26 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:17.449942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67bc9-285f-4dad-b876-3f891c52ab26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:17.451726 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6bc67bc9-285f-4dad-b876-3f891c52ab26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:17.463897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc67bc9-285f-4dad-b876-3f891c52ab26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:17.470181 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc67bc9-285f-4dad-b876-3f891c52ab26 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:17.481456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1393/2770] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:23.615533 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=28 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:27.473349 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=52 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:40.221334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fd501-1c15-49a9-8c26-ef629d2a6ecb None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:40.233642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fd501-1c15-49a9-8c26-ef629d2a6ecb None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:40.238212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:40.239095 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:40.239385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:40.239668 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] PATH_INFO: `/v3/users/f4cc8e2a600144fe9a0a65cb8f2c5b83` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:40.241723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4cc8e2a600144fe9a0a65cb8f2c5b83)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:40.244825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:40.478679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4cc8e2a600144fe9a0a65cb8f2c5b83'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:40.487052 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4cc8e2a600144fe9a0a65cb8f2c5b83'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:40.538151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4cc8e2a600144fe9a0a65cb8f2c5b83'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:40.538773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4cc8e2a600144fe9a0a65cb8f2c5b83'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:40.540004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4cc8e2a600144fe9a0a65cb8f2c5b83'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:40.557530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1378/2771] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:40 2022] DELETE /identity/v3/users/f4cc8e2a600144fe9a0a65cb8f2c5b83 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:40.562026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:40.562480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:40.562576 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:40.579934 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:40.581169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:40.641781 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1394/2772] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:40.651776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:40.653226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:40.724620 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fea79-0a7b-4c03-b66d-b7098da8f1ef admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:40.728855 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a771025-1d12-4ebf-b350-779020138e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:40.729536 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771025-1d12-4ebf-b350-779020138e23 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:40.729821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771025-1d12-4ebf-b350-779020138e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:40.730096 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a771025-1d12-4ebf-b350-779020138e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:40.739328 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a771025-1d12-4ebf-b350-779020138e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:40.740801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a771025-1d12-4ebf-b350-779020138e23 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:40.750506 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1379/2773] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:41.140243 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:41.142134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:41.163338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6ad52-c067-4ffd-ae80-043f6f6e6414 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:41.171913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:41.174074 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.174445 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.174801 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] PATH_INFO: `/v3/projects/4a434570640b49848a9c3020208efcab` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.176588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a434570640b49848a9c3020208efcab)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:41.178849 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:41.287895 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a434570640b49848a9c3020208efcab', 'request_id': 'req-fb36b7ec-3e28-4341-a167-b6173042f2ee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:41.295056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a434570640b49848a9c3020208efcab', 'request_id': 'req-fb36b7ec-3e28-4341-a167-b6173042f2ee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:41.301175 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a434570640b49848a9c3020208efcab', 'request_id': 'req-fb36b7ec-3e28-4341-a167-b6173042f2ee'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:41.360695 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] The token cache is being invalidate because project 4a434570640b49848a9c3020208efcab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:41.360993 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:41.361627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-fb36b7ec-3e28-4341-a167-b6173042f2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:41.363193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1395/2774] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:41 2022] DELETE /identity/v3/projects/4a434570640b49848a9c3020208efcab => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:41.368124 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737c93b-544d-4732-8318-6d37d3d30a79 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.368629 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737c93b-544d-4732-8318-6d37d3d30a79 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.371905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737c93b-544d-4732-8318-6d37d3d30a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.386991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6737c93b-544d-4732-8318-6d37d3d30a79 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:41.388290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6737c93b-544d-4732-8318-6d37d3d30a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:41.451999 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1380/2775] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:41.459609 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeaed8-3f7a-4cfe-8148-85ebede93957 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.459896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeaed8-3f7a-4cfe-8148-85ebede93957 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.460185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeaed8-3f7a-4cfe-8148-85ebede93957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.479407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-1ebeaed8-3f7a-4cfe-8148-85ebede93957 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:41.480669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-1ebeaed8-3f7a-4cfe-8148-85ebede93957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:41.488650 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1396/2776] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:41.492813 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737c93b-544d-4732-8318-6d37d3d30a79 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:41.502944 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737c93b-544d-4732-8318-6d37d3d30a79 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:41.507524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:41.508762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.509984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.509984 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.510198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:55:41.510376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:41.512006 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:41.528624 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1381/2777] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 17:55:41.530461 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeaed8-3f7a-4cfe-8148-85ebede93957 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:41.539904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeaed8-3f7a-4cfe-8148-85ebede93957 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:41.544677 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:41.545335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.545622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.545902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.546603 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:41.548521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:41.619509 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1397/2778] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:41 2022] POST /identity/v3/projects => generated 391 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:41.624062 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:41.629447 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:41.635242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11bd16-5010-411b-9004-1c7118fbd767 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:41.692864 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:41.693551 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.694245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.694514 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.695712 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:41.697745 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:41.784089 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1382/2779] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:41 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:41.788111 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:41.798531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9706c9-9076-4b89-9e01-79384879785d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:41.803193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:41.803845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.804132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.804410 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.805179 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:41.806774 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:41.817360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1398/2780] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:41.821810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:41.831770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364388b-f1d0-461f-99dc-22c115cc8cb1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:41.837220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0e1cb-23e3-4c8a-bc96-12be7dc37c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:41.837880 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0e1cb-23e3-4c8a-bc96-12be7dc37c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.838176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0e1cb-23e3-4c8a-bc96-12be7dc37c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.838458 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0e1cb-23e3-4c8a-bc96-12be7dc37c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.839179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0e1cb-23e3-4c8a-bc96-12be7dc37c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:41.840821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0e1cb-23e3-4c8a-bc96-12be7dc37c7b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:41.850682 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1383/2781] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:41.854590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:41.864539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20082335-07e5-4086-be6f-a0b0d6792f7b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:41.874598 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:41.875498 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:41.875790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:41.876073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] PATH_INFO: `/v3/projects/99b31cb3018f4fa3a0acaba93893f128/users/a522daff189f4e6dbcf419deace704d6/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:41.893293 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b31cb3018f4fa3a0acaba93893f128, user_id=a522daff189f4e6dbcf419deace704d6, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:41.895987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:41.989911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1399/2782] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:41 2022] PUT /identity/v3/projects/99b31cb3018f4fa3a0acaba93893f128/users/a522daff189f4e6dbcf419deace704d6/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:42.001837 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:42.002133 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:42.002416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:42.029966 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] MFA Rules not processed for user `a522daff189f4e6dbcf419deace704d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:42.031289 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:42.094449 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1384/2783] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:42.105636 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:42.107291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:42.129067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:42.164622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae6bf1-cc70-469a-8e17-e55fbbf871f2 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:42.168294 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:42.168963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:42.169246 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:42.169520 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:42.178024 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:42.179482 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:42.188890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1400/2784] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:44.823943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:44.825518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:44.892125 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efd594-d86e-49f8-9096-19f0766171ac None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:44.896004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dbbf2-6c60-4580-95ce-f8a8523f1a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:44.896709 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbbf2-6c60-4580-95ce-f8a8523f1a90 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:44.897192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbbf2-6c60-4580-95ce-f8a8523f1a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:44.897324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbbf2-6c60-4580-95ce-f8a8523f1a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:44.899218 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a69dbbf2-6c60-4580-95ce-f8a8523f1a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:44.927710 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:44.964296 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dbbf2-6c60-4580-95ce-f8a8523f1a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:44.965785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dbbf2-6c60-4580-95ce-f8a8523f1a90 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:44.976767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1385/2785] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:49.357682 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:49.359476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:49.437026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee754760-d040-403f-bee2-0a662d5c5ad4 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:49.441539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:49.442414 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:49.442696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:49.442972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] PATH_INFO: `/v3/users/f5234efe1b154f5bb064cf3996a96a31` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:49.444300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5234efe1b154f5bb064cf3996a96a31)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:49.446301 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:49.721491 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5234efe1b154f5bb064cf3996a96a31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:49.742139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5234efe1b154f5bb064cf3996a96a31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:49.742910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5234efe1b154f5bb064cf3996a96a31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:49.751057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5234efe1b154f5bb064cf3996a96a31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:49.751659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5234efe1b154f5bb064cf3996a96a31'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:49.846905 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1401/2786] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:49 2022] DELETE /identity/v3/users/f5234efe1b154f5bb064cf3996a96a31 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:49.851568 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:49.852042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:49.852154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:49.870038 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:49.872983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:49.941426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1386/2787] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:49.950640 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:49.952073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:49.999960 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:50.012104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600c400-f459-41f6-81a6-8d1e525b036d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:50.015656 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:50.016314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:50.016624 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:50.016914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:50.026754 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:50.029115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:50.044912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1402/2788] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:50.431140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:50.433262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:50.445065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d4b3c-d6a5-4b65-a214-fa14c9aa039e None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:50.454695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:50.457906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:50.458205 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:50.458495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] PATH_INFO: `/v3/projects/632afe8a74304c8cb5d65135bd212ed5` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:50.460426 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] RBAC: Authorizing `identity:delete_project(project_id=632afe8a74304c8cb5d65135bd212ed5)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:50.463301 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:50.561031 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '632afe8a74304c8cb5d65135bd212ed5', 'request_id': 'req-ad3a9172-5459-402d-a259-08fce0bd479a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.567339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '632afe8a74304c8cb5d65135bd212ed5', 'request_id': 'req-ad3a9172-5459-402d-a259-08fce0bd479a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.646796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '632afe8a74304c8cb5d65135bd212ed5', 'request_id': 'req-ad3a9172-5459-402d-a259-08fce0bd479a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.658943 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] The token cache is being invalidate because project 632afe8a74304c8cb5d65135bd212ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:50.659242 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.659889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.661678 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1387/2789] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:50 2022] DELETE /identity/v3/projects/632afe8a74304c8cb5d65135bd212ed5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:50.665538 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:50.675516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443d142-c238-45b9-883b-b16be9ff93e3 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:50.736618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:50.737580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:50.737931 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:50.738242 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] PATH_INFO: `/v3/users/c6a99144bfbd4812a92bf2a251364383` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:50.739443 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a99144bfbd4812a92bf2a251364383)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:50.741426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:50.906040 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a99144bfbd4812a92bf2a251364383'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.918214 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a99144bfbd4812a92bf2a251364383'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.918912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a99144bfbd4812a92bf2a251364383'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.925872 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a99144bfbd4812a92bf2a251364383'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.926550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6d55dcac-7e27-45eb-878a-b9cba003f5aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a99144bfbd4812a92bf2a251364383'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:50.983570 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1403/2790] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:50 2022] DELETE /identity/v3/users/c6a99144bfbd4812a92bf2a251364383 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:51.396140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:51.397708 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:51.412132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a9172-5459-402d-a259-08fce0bd479a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:51.458150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:51.469694 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:51.470564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:51.470853 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:51.471136 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] PATH_INFO: `/v3/projects/dac767e74d3540d4a407fc9e12e18be4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:51.472323 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac767e74d3540d4a407fc9e12e18be4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:51.474298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:51.608290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac767e74d3540d4a407fc9e12e18be4', 'request_id': 'req-a88a9485-ce92-4528-adf4-c395d6959c03'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:51.614557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac767e74d3540d4a407fc9e12e18be4', 'request_id': 'req-a88a9485-ce92-4528-adf4-c395d6959c03'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:51.692083 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac767e74d3540d4a407fc9e12e18be4', 'request_id': 'req-a88a9485-ce92-4528-adf4-c395d6959c03'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:51.703130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] The token cache is being invalidate because project dac767e74d3540d4a407fc9e12e18be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:51.703417 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:51.704045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-a88a9485-ce92-4528-adf4-c395d6959c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:51.705680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1388/2791] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:51 2022] DELETE /identity/v3/projects/dac767e74d3540d4a407fc9e12e18be4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:51.710523 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa13c8-112a-4b7e-a4dc-490afd2e2a63 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:51.710935 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa13c8-112a-4b7e-a4dc-490afd2e2a63 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:51.711149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa13c8-112a-4b7e-a4dc-490afd2e2a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:51.730713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-dbaa13c8-112a-4b7e-a4dc-490afd2e2a63 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:51.731984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-dbaa13c8-112a-4b7e-a4dc-490afd2e2a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:51.793925 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1404/2792] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:51.802594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbeca-c7dc-4749-936a-bece23375a82 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:51.802888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbeca-c7dc-4749-936a-bece23375a82 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:51.803173 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbeca-c7dc-4749-936a-bece23375a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:51.821621 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-863fbeca-c7dc-4749-936a-bece23375a82 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:51.822835 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-863fbeca-c7dc-4749-936a-bece23375a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:51.830801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1389/2793] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:51.835056 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa13c8-112a-4b7e-a4dc-490afd2e2a63 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:51.841808 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:51.845871 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa13c8-112a-4b7e-a4dc-490afd2e2a63 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:51.850742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:51.851524 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:51.851842 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:51.852120 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:51.852939 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:55:51.853984 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:51.855456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:51.868148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1405/2794] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:55:51.872877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fbeca-c7dc-4749-936a-bece23375a82 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:51.889416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fbeca-c7dc-4749-936a-bece23375a82 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:51.894237 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:51.894895 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:51.895181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:51.895457 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:51.896187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:51.898141 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.000488 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1390/2795] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:51 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:52.001824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:52.012809 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cb2fe-1d29-4784-b5c8-85612a89e0bd admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:52.071309 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:52.072517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.072824 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.073098 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.073790 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:52.075702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.150792 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1406/2796] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:52 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:52.155209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:52.165494 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82149c5-4211-4f04-beb2-19d4aed43d4f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:52.170171 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:52.170836 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.171123 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.171406 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.172134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:52.173798 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.184217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1391/2797] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:52.188163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:52.198962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214915e-4785-4108-aeb9-8cdd64363dfb admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:52.203678 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95dc41-da96-458d-9c95-058a68310067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:52.208185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95dc41-da96-458d-9c95-058a68310067 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.208185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95dc41-da96-458d-9c95-058a68310067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.208185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95dc41-da96-458d-9c95-058a68310067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.208185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95dc41-da96-458d-9c95-058a68310067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:52.208185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95dc41-da96-458d-9c95-058a68310067 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.218416 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1407/2798] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:52.222508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:52.236983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a062ecf-d192-42f8-9ddc-188637d972ad admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:52.242283 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:52.243190 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.243484 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.243767 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] PATH_INFO: `/v3/projects/e00fba6fbfc64f2ca558f1a1c085df7f/users/6add1f1b51954c06818201eccbd781e1/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.265192 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00fba6fbfc64f2ca558f1a1c085df7f, user_id=6add1f1b51954c06818201eccbd781e1, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:52.269033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.352009 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1392/2799] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:52 2022] PUT /identity/v3/projects/e00fba6fbfc64f2ca558f1a1c085df7f/users/6add1f1b51954c06818201eccbd781e1/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:52.362515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed1d42-429c-4d3a-9fa5-87388f2b6df7 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.362810 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed1d42-429c-4d3a-9fa5-87388f2b6df7 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.363115 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed1d42-429c-4d3a-9fa5-87388f2b6df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.393184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-28ed1d42-429c-4d3a-9fa5-87388f2b6df7 None None] MFA Rules not processed for user `6add1f1b51954c06818201eccbd781e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:52.394531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-28ed1d42-429c-4d3a-9fa5-87388f2b6df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:52.513102 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1408/2800] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:52.518046 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:52.585430 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbd15c-765c-4d27-8836-82a1a7b208e5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:52.591300 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:52.592012 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.592526 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.592843 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.593683 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:52.596041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.659671 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17710c9-8a76-4c2c-80af-4eb15f0b21ec None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.661068 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17710c9-8a76-4c2c-80af-4eb15f0b21ec None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.661361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17710c9-8a76-4c2c-80af-4eb15f0b21ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.678889 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:52.683348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-a17710c9-8a76-4c2c-80af-4eb15f0b21ec None None] MFA Rules not processed for user `62fbd947e2a14d11b5cfeb26730347e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:52.684592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a17710c9-8a76-4c2c-80af-4eb15f0b21ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:52.692035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1393/2801] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:52 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:52.693555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:52.759996 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1409/2802] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:52.779972 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c1638-cee8-467a-b5cd-3d4474975451 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:52.788815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:52.789781 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.790181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.791219 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.792015 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:52.794011 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.942586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1394/2803] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:52 2022] POST /identity/v3/users => generated 299 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:52.944571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17710c9-8a76-4c2c-80af-4eb15f0b21ec None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:52.953944 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:52.959517 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17710c9-8a76-4c2c-80af-4eb15f0b21ec None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:52.965118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:52.966812 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:52.967403 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:52.967902 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:52.968920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:52.970932 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:52.984060 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1410/2804] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:55:52.988209 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:53.007045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6d296-d526-48fb-8924-dfb7e46ee3b6 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:53.012958 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:53.013994 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:53.017004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:53.017004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] PATH_INFO: `/v3/projects/f6b05309be1c4c5089b877c96a1755c4/users/3a08664524864fe1a070965a4e613642/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:53.037754 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b05309be1c4c5089b877c96a1755c4, user_id=3a08664524864fe1a070965a4e613642, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:53.045812 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:53.158970 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1395/2805] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:52 2022] PUT /identity/v3/projects/f6b05309be1c4c5089b877c96a1755c4/users/3a08664524864fe1a070965a4e613642/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:53.162923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:53.187744 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5b8a6-ee31-4a87-bac3-ff67cbe05bab admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:53.272884 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9da4c5-ca73-4f74-8c26-8d5466cd4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:53.273565 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da4c5-ca73-4f74-8c26-8d5466cd4735 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:53.273903 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da4c5-ca73-4f74-8c26-8d5466cd4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:53.274136 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9da4c5-ca73-4f74-8c26-8d5466cd4735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:53.275387 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9da4c5-ca73-4f74-8c26-8d5466cd4735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:53.277180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9da4c5-ca73-4f74-8c26-8d5466cd4735 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:53.289038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1411/2806] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:53.293801 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:53.304265 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d609921-c07c-4ed9-b190-87e43a7fc4c5 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:53.309201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:53.310085 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:53.310376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:53.310655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] PATH_INFO: `/v3/projects/f6b05309be1c4c5089b877c96a1755c4/users/3a08664524864fe1a070965a4e613642/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:53.312659 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b05309be1c4c5089b877c96a1755c4, user_id=3a08664524864fe1a070965a4e613642, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:53.315351 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:53.327147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1396/2807] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:53 2022] PUT /identity/v3/projects/f6b05309be1c4c5089b877c96a1755c4/users/3a08664524864fe1a070965a4e613642/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:53.338112 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe668c5-710d-49bc-9949-46cfaefa0ec8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:53.339913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe668c5-710d-49bc-9949-46cfaefa0ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:53.339913 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe668c5-710d-49bc-9949-46cfaefa0ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:53.369823 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-3fe668c5-710d-49bc-9949-46cfaefa0ec8 None None] MFA Rules not processed for user `3a08664524864fe1a070965a4e613642`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:53.372103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-3fe668c5-710d-49bc-9949-46cfaefa0ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:53.443281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1412/2808] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:53.507026 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:53.512480 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:53.622020 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22940956-9635-49d8-8440-0a5cfd0a1300 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:53.637513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a13854-a503-460b-9536-8d6719a8f825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:53.638245 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13854-a503-460b-9536-8d6719a8f825 service cinder] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:53.638545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13854-a503-460b-9536-8d6719a8f825 service cinder] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:53.639723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a13854-a503-460b-9536-8d6719a8f825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:53.651637 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a13854-a503-460b-9536-8d6719a8f825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:53.654043 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a13854-a503-460b-9536-8d6719a8f825 service cinder] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:53.666114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1397/2809] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:55:53.698476 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe668c5-710d-49bc-9949-46cfaefa0ec8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:53.790264 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe668c5-710d-49bc-9949-46cfaefa0ec8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:53.800104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:53.801031 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:53.803016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:53.803016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] PATH_INFO: `/v3/users/6add1f1b51954c06818201eccbd781e1` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:53.803016 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6add1f1b51954c06818201eccbd781e1)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:53.804846 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:53.822740 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:53.999583 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6add1f1b51954c06818201eccbd781e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.012797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6add1f1b51954c06818201eccbd781e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.013428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6add1f1b51954c06818201eccbd781e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.020646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6add1f1b51954c06818201eccbd781e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.021311 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6add1f1b51954c06818201eccbd781e1'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.098296 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1413/2810] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:53 2022] DELETE /identity/v3/users/6add1f1b51954c06818201eccbd781e1 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:54.103513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:54.103513 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:54.103802 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:54.123008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:54.123955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:54.174150 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:54.187114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1398/2811] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:55:54.197789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:54.199215 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:54.262449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeadb8-57ff-49ae-8ef6-dc20bc46bd1f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:54.266633 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:54.267307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:54.267767 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:54.267988 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:54.277981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:54.279550 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:54.290229 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1414/2812] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:54.758435 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:54.760148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:54.773583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1cb15-3cf7-4c08-9078-541b52c2bfd0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:54.780776 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:54.781815 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:54.782102 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:54.782388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] PATH_INFO: `/v3/projects/e00fba6fbfc64f2ca558f1a1c085df7f` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:54.783586 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00fba6fbfc64f2ca558f1a1c085df7f)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:54.785608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:54.870091 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00fba6fbfc64f2ca558f1a1c085df7f', 'request_id': 'req-235e6fcb-96f8-483e-a16e-b72a9b90f3db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.876905 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00fba6fbfc64f2ca558f1a1c085df7f', 'request_id': 'req-235e6fcb-96f8-483e-a16e-b72a9b90f3db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.920562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00fba6fbfc64f2ca558f1a1c085df7f', 'request_id': 'req-235e6fcb-96f8-483e-a16e-b72a9b90f3db'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.931084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] The token cache is being invalidate because project e00fba6fbfc64f2ca558f1a1c085df7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:54.931369 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.932004 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:54.933576 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1399/2813] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:54 2022] DELETE /identity/v3/projects/e00fba6fbfc64f2ca558f1a1c085df7f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:54.937862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:54.948411 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6a154-279f-40db-988d-b1e205133da1 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:55.004741 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:55.005611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:55.005897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:55.006174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] PATH_INFO: `/v3/users/3a08664524864fe1a070965a4e613642` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:55.007354 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a08664524864fe1a070965a4e613642)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:55.009868 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:55.209157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:55.216276 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a08664524864fe1a070965a4e613642'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:55.228106 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a08664524864fe1a070965a4e613642'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:55.228862 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a08664524864fe1a070965a4e613642'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:55.235825 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a08664524864fe1a070965a4e613642'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:55.236463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-afd27fd0-0cd5-44c4-bd74-64711939a49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a08664524864fe1a070965a4e613642'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:55.315845 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1415/2814] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:55:54 2022] DELETE /identity/v3/users/3a08664524864fe1a070965a4e613642 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:56.019598 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:56.021107 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:56.034592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e6fcb-96f8-483e-a16e-b72a9b90f3db admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:56.093224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:56.094086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.094371 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.094676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] PATH_INFO: `/v3/projects/f6b05309be1c4c5089b877c96a1755c4` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.095877 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b05309be1c4c5089b877c96a1755c4)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:56.097892 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:56.231177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b05309be1c4c5089b877c96a1755c4', 'request_id': 'req-48cfcbed-54ee-4dd2-912b-8d61300c7427'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:56.237446 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b05309be1c4c5089b877c96a1755c4', 'request_id': 'req-48cfcbed-54ee-4dd2-912b-8d61300c7427'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:56.303580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b05309be1c4c5089b877c96a1755c4', 'request_id': 'req-48cfcbed-54ee-4dd2-912b-8d61300c7427'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:56.315384 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] The token cache is being invalidate because project f6b05309be1c4c5089b877c96a1755c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:55:56.315674 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:56.316298 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-48cfcbed-54ee-4dd2-912b-8d61300c7427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:55:56.318423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1400/2815] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:55:56 2022] DELETE /identity/v3/projects/f6b05309be1c4c5089b877c96a1755c4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:55:56.323559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a5771-45a6-4295-b3fe-5d6255d45f2b None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.324009 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a5771-45a6-4295-b3fe-5d6255d45f2b None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.324107 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a5771-45a6-4295-b3fe-5d6255d45f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.342132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f23a5771-45a6-4295-b3fe-5d6255d45f2b None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:56.343335 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f23a5771-45a6-4295-b3fe-5d6255d45f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:56.402617 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1416/2816] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:56.411282 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e6851-12c5-444e-87cc-74f8415af033 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.411583 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e6851-12c5-444e-87cc-74f8415af033 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.411869 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e6851-12c5-444e-87cc-74f8415af033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.430596 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cf0e6851-12c5-444e-87cc-74f8415af033 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:56.431807 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cf0e6851-12c5-444e-87cc-74f8415af033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:56.439691 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1401/2817] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:56.443783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a5771-45a6-4295-b3fe-5d6255d45f2b None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:56.454034 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a5771-45a6-4295-b3fe-5d6255d45f2b None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:56.458763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:56.459426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.459718 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.460004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.460789 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:55:56.461129 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:56.463375 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:56.475185 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1417/2818] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:55:56.479496 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e6851-12c5-444e-87cc-74f8415af033 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:56.483846 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:56.489036 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e6851-12c5-444e-87cc-74f8415af033 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:56.494642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:56.494642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.494642 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.494992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.495689 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:56.497931 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:56.614399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1402/2819] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:55:56 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:56.618611 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:56.630073 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a340acc-86bb-4b2a-8420-8ceecc105d51 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:56.706318 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:56.707038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.707340 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.707618 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.708325 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:56.710297 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:56.825257 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1418/2820] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:55:56 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:55:56.830262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:56.840564 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272d884-df2d-4a9a-8d00-5e5c90ef7b2e admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:56.845092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:56.845792 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.846074 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.846436 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.847161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:56.848833 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:56.862010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1403/2821] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:56.863163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:56.874163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d9584-add0-4da2-be9a-83b64abe61f1 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:56.885077 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a7146-cc25-4bce-a97a-1a227bdd5c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:56.885766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a7146-cc25-4bce-a97a-1a227bdd5c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.891270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a7146-cc25-4bce-a97a-1a227bdd5c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.891270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a7146-cc25-4bce-a97a-1a227bdd5c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.891270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a7146-cc25-4bce-a97a-1a227bdd5c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:56.891270 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a7146-cc25-4bce-a97a-1a227bdd5c36 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:56.899426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1419/2822] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:55:56.903738 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:56.923165 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df62a-8de3-4a6b-902d-d0c3d4ea7c94 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:56.928207 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:56.929206 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:56.929558 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:56.929906 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] PATH_INFO: `/v3/projects/c50910a24d8e44698699da6688245497/users/174a0a6001a648119ebe3fda0292d1c7/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:56.956796 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50910a24d8e44698699da6688245497, user_id=174a0a6001a648119ebe3fda0292d1c7, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:56.960170 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:57.018886 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1404/2823] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:55:56 2022] PUT /identity/v3/projects/c50910a24d8e44698699da6688245497/users/174a0a6001a648119ebe3fda0292d1c7/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:55:57.029733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:57.030029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:57.030338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:57.056976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] MFA Rules not processed for user `174a0a6001a648119ebe3fda0292d1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:55:57.058197 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:57.067412 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:55:57.129067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1420/2824] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:55:57.140850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:57.142705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:57.200823 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b248ca5-3683-4b9e-a650-28ac5cc04db9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:57.204519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050182da-2561-4657-a574-333899a1094f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b0f154abf64d57a431b956a1822300', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:57.205198 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050182da-2561-4657-a574-333899a1094f service glance] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:57.205534 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050182da-2561-4657-a574-333899a1094f service glance] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:57.205816 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050182da-2561-4657-a574-333899a1094f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:57.218600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050182da-2561-4657-a574-333899a1094f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:57.220082 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050182da-2561-4657-a574-333899a1094f service glance] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:57.232993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1405/2825] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:55:58.515020 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:55:58.516495 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:58.605963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c9d82-37b3-4e4b-a96f-9f5daf7aad6c None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:55:58.610945 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc59e2-49c9-47e6-962b-f7dcde84a110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:55:58.612036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc59e2-49c9-47e6-962b-f7dcde84a110 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:55:58.612555 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc59e2-49c9-47e6-962b-f7dcde84a110 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:55:58.612877 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc59e2-49c9-47e6-962b-f7dcde84a110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:55:58.615046 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-7dfc59e2-49c9-47e6-962b-f7dcde84a110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:55:58.712691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc59e2-49c9-47e6-962b-f7dcde84a110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:55:58.714480 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc59e2-49c9-47e6-962b-f7dcde84a110 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:55:58.727088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1421/2826] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:56:03.646964 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050182da-2561-4657-a574-333899a1094f service glance] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:03.648522 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-050182da-2561-4657-a574-333899a1094f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:03.663147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050182da-2561-4657-a574-333899a1094f service glance] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:03.667698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:03.668577 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:03.673581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:03.673581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] PATH_INFO: `/v3/users/a522daff189f4e6dbcf419deace704d6` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:03.673581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a522daff189f4e6dbcf419deace704d6)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:03.673581 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:03.931560 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a522daff189f4e6dbcf419deace704d6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:03.938930 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a522daff189f4e6dbcf419deace704d6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:04.012825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a522daff189f4e6dbcf419deace704d6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:04.014666 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a522daff189f4e6dbcf419deace704d6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:04.015455 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a522daff189f4e6dbcf419deace704d6'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:04.058362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1406/2827] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:56:03 2022] DELETE /identity/v3/users/a522daff189f4e6dbcf419deace704d6 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 17:56:04.059783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:04.060266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:04.060658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:04.080955 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:04.082702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:04.221696 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1422/2828] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:56:04.256356 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:04.257941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:04.284299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:04.337103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685da1b4-dc60-4031-a058-af8de03324c9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:04.345010 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:04.345782 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:04.347377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:04.347729 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:04.365054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:04.366579 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:04.385084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1407/2829] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:56:04.883434 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:04.885315 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:04.897126 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=100 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:04.911110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe918cd-0152-4be7-859e-b1aa50789b3d None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:04.921864 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:04.923235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:04.923974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:04.928516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] PATH_INFO: `/v3/projects/99b31cb3018f4fa3a0acaba93893f128` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:04.929829 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b31cb3018f4fa3a0acaba93893f128)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:04.932836 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:05.125273 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b31cb3018f4fa3a0acaba93893f128', 'request_id': 'req-b200317e-328b-4685-8598-6d60f4ac7f2d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:05.132979 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b31cb3018f4fa3a0acaba93893f128', 'request_id': 'req-b200317e-328b-4685-8598-6d60f4ac7f2d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:05.139588 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b31cb3018f4fa3a0acaba93893f128', 'request_id': 'req-b200317e-328b-4685-8598-6d60f4ac7f2d'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:05.235732 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] The token cache is being invalidate because project 99b31cb3018f4fa3a0acaba93893f128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:56:05.236174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:05.238366 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:05.253727 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1423/2830] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:56:04 2022] DELETE /identity/v3/projects/99b31cb3018f4fa3a0acaba93893f128 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:07.548385 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:07.549900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:07.569454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac5a72-7edd-4e4f-a845-c0c843c328af service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:07.628993 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:07.629887 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:07.630204 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:07.633345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] PATH_INFO: `/v3/users/e87a32eb7afd4eb5ae5c6cdbf06c65d1` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:07.635181 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e87a32eb7afd4eb5ae5c6cdbf06c65d1)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:07.637324 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:07.844764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e87a32eb7afd4eb5ae5c6cdbf06c65d1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:07.855503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e87a32eb7afd4eb5ae5c6cdbf06c65d1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:07.910002 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e87a32eb7afd4eb5ae5c6cdbf06c65d1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:07.910704 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e87a32eb7afd4eb5ae5c6cdbf06c65d1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:07.911727 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e87a32eb7afd4eb5ae5c6cdbf06c65d1'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:07.932920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1408/2831] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:56:07 2022] DELETE /identity/v3/users/e87a32eb7afd4eb5ae5c6cdbf06c65d1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:08.196797 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:08.199171 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:08.221384 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200317e-328b-4685-8598-6d60f4ac7f2d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:08.288447 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:08.289419 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:08.289711 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:08.290589 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] PATH_INFO: `/v3/projects/8d4d85b85edb4aa3b18b88ae45c3b702` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:08.291814 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4d85b85edb4aa3b18b88ae45c3b702)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:08.293976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:08.436571 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4d85b85edb4aa3b18b88ae45c3b702', 'request_id': 'req-f6374e93-08f6-46d2-af12-adc366080328'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.446769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4d85b85edb4aa3b18b88ae45c3b702', 'request_id': 'req-f6374e93-08f6-46d2-af12-adc366080328'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.454994 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4d85b85edb4aa3b18b88ae45c3b702', 'request_id': 'req-f6374e93-08f6-46d2-af12-adc366080328'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.503642 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] The token cache is being invalidate because project 8d4d85b85edb4aa3b18b88ae45c3b702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:56:08.504029 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.504799 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.506958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1424/2832] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:56:08 2022] DELETE /identity/v3/projects/8d4d85b85edb4aa3b18b88ae45c3b702 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:08.511698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:08.513888 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:08.528244 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a462b3-09b9-4691-ab32-5c0067611e2f admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:08.569857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:08.586153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:08.587661 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:08.587945 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:08.588224 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] PATH_INFO: `/v3/users/29ecf13bb4d647ed9ce6396f8a072ad3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:08.589478 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ecf13bb4d647ed9ce6396f8a072ad3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:08.591416 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:08.735391 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ecf13bb4d647ed9ce6396f8a072ad3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.742104 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ecf13bb4d647ed9ce6396f8a072ad3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.807013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ecf13bb4d647ed9ce6396f8a072ad3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.807635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ecf13bb4d647ed9ce6396f8a072ad3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.808251 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ecf13bb4d647ed9ce6396f8a072ad3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:08.826778 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1409/2833] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:56:08 2022] DELETE /identity/v3/users/29ecf13bb4d647ed9ce6396f8a072ad3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:09.079501 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:09.081084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:09.099124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374e93-08f6-46d2-af12-adc366080328 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:09.124556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:09.173703 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:09.174679 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:09.175241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:09.175456 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] PATH_INFO: `/v3/projects/f1f01c4526b74cd19bb67a4e1ea98c5c` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:09.176910 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f01c4526b74cd19bb67a4e1ea98c5c)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:09.178866 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:09.315983 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f01c4526b74cd19bb67a4e1ea98c5c', 'request_id': 'req-70b3940b-fac0-4823-86a2-587992dec10b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:09.322163 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f01c4526b74cd19bb67a4e1ea98c5c', 'request_id': 'req-70b3940b-fac0-4823-86a2-587992dec10b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:09.327572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f01c4526b74cd19bb67a4e1ea98c5c', 'request_id': 'req-70b3940b-fac0-4823-86a2-587992dec10b'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:09.411858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] The token cache is being invalidate because project f1f01c4526b74cd19bb67a4e1ea98c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:56:09.412373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:09.413021 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-70b3940b-fac0-4823-86a2-587992dec10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:09.414590 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1425/2834] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:56:09 2022] DELETE /identity/v3/projects/f1f01c4526b74cd19bb67a4e1ea98c5c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:18.776612 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:18.778044 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:18.791473 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef9615-b815-4c43-97f2-c925c8a9b3a1 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:18.847737 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:18.848645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:18.848941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:18.849229 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] PATH_INFO: `/v3/users/174a0a6001a648119ebe3fda0292d1c7` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:18.850454 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=174a0a6001a648119ebe3fda0292d1c7)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:18.852408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:18.994409 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:18.995890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:18.995890 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:19.013947 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:19.015139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:19.054127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174a0a6001a648119ebe3fda0292d1c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:19.061065 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174a0a6001a648119ebe3fda0292d1c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:19.078719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1426/2835] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Jul 15 17:56:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:56:19.121712 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:19.123395 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:19.137290 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174a0a6001a648119ebe3fda0292d1c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:19.138156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174a0a6001a648119ebe3fda0292d1c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:19.138794 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-bfc23f53-8e3a-4607-b8f2-62a91dba93d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174a0a6001a648119ebe3fda0292d1c7'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:19.140211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:19.158103 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1410/2836] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:56:18 2022] DELETE /identity/v3/users/174a0a6001a648119ebe3fda0292d1c7 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:19.162516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:19.163018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:19.163307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:19.172078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:19.183013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:19.184334 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:19.198659 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9864b1-c112-459a-8422-e1f3cd29eba8 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:19.248870 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1411/2837] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:56:19.259910 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:19.265917 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:19.266067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:19.266067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:19.266067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:19.266067 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:19.266457 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:19.321236 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3ad5-2be1-45b6-853e-64703f27191a None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:19.325567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b54a8-10e5-43a5-81ca-e2447f48b117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:19.325769 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:19.326220 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b54a8-10e5-43a5-81ca-e2447f48b117 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:19.326508 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b54a8-10e5-43a5-81ca-e2447f48b117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:19.326789 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b54a8-10e5-43a5-81ca-e2447f48b117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:19.327388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:19.335776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1427/2838] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:56:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 17:56:19.337169 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b54a8-10e5-43a5-81ca-e2447f48b117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:19.339785 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b54a8-10e5-43a5-81ca-e2447f48b117 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:19.352921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1412/2839] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 17:56:19.839385 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:19.839922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:19.878858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f87a48-0cab-488e-863a-a234d0d1cdce service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:19.887290 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:19.891002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:19.891002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:19.891002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] PATH_INFO: `/v3/projects/c50910a24d8e44698699da6688245497` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:19.891002 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c50910a24d8e44698699da6688245497)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:19.891985 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:20.030052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c50910a24d8e44698699da6688245497', 'request_id': 'req-7ccfc07b-3c48-4a52-9886-5d8091721496'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:20.043605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c50910a24d8e44698699da6688245497', 'request_id': 'req-7ccfc07b-3c48-4a52-9886-5d8091721496'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:20.052920 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c50910a24d8e44698699da6688245497', 'request_id': 'req-7ccfc07b-3c48-4a52-9886-5d8091721496'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:20.166876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] The token cache is being invalidate because project c50910a24d8e44698699da6688245497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:56:20.167164 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:20.167853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-7ccfc07b-3c48-4a52-9886-5d8091721496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:20.169559 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1428/2840] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:56:19 2022] DELETE /identity/v3/projects/c50910a24d8e44698699da6688245497 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:20.176954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a9b97-f455-4c4d-8220-9f3adb14e1d2 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.177512 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a9b97-f455-4c4d-8220-9f3adb14e1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.177625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a9b97-f455-4c4d-8220-9f3adb14e1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.212153 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-cd2a9b97-f455-4c4d-8220-9f3adb14e1d2 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:20.217001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-cd2a9b97-f455-4c4d-8220-9f3adb14e1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:20.298987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1413/2841] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:20.307951 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1424c-aa8c-43e9-90f9-edd85ef866ef None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.308265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1424c-aa8c-43e9-90f9-edd85ef866ef None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.308627 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1424c-aa8c-43e9-90f9-edd85ef866ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.331915 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-94f1424c-aa8c-43e9-90f9-edd85ef866ef None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:20.333660 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-94f1424c-aa8c-43e9-90f9-edd85ef866ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:20.342521 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1429/2842] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:20.347223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a9b97-f455-4c4d-8220-9f3adb14e1d2 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:20.358071 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a9b97-f455-4c4d-8220-9f3adb14e1d2 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:20.365081 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:20.366067 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.366357 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.366643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.367414 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:56:20.367728 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:20.370001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:20.388505 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1414/2843] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:56:20.389844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1424c-aa8c-43e9-90f9-edd85ef866ef None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:20.398344 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:20.402912 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1424c-aa8c-43e9-90f9-edd85ef866ef None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:20.408666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:20.409345 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.411038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.411038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] PATH_INFO: `/v3/projects` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.411038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:20.412770 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:20.513530 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:20.513749 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1430/2844] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:56:20 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:56:20.527106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0c850-fe20-4d9d-b5d8-abe8bac044bd admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:20.538566 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:20.603293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:20.603986 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.604273 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.604711 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] PATH_INFO: `/v3/users` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.605466 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:20.608018 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:20.702592 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1415/2845] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:56:20 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:56:20.707007 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:20.718462 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f1c61-d74a-48d9-8974-cf1921c3d07f admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:20.723425 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:20.724104 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.724394 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.724721 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.725450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:20.728044 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:20.745958 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1431/2846] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 17:56:20.747545 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:20.761178 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da950bfa-0048-42db-98d9-54791d3c6f85 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:20.769703 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15a07c-34ef-4805-ad80-4933e06bb2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:20.771540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15a07c-34ef-4805-ad80-4933e06bb2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.772176 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15a07c-34ef-4805-ad80-4933e06bb2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.773088 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15a07c-34ef-4805-ad80-4933e06bb2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.774561 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15a07c-34ef-4805-ad80-4933e06bb2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:20.782793 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15a07c-34ef-4805-ad80-4933e06bb2e0 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:20.800679 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1416/2847] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:56:20.803210 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:20.815548 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9a9d4-568a-4a5e-8b0a-3cc9e6910e5b admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:20.820933 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:20.821963 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.822388 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.822778 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] PATH_INFO: `/v3/projects/a55c434befac4454b8bd21cf3053f3e2/users/e86c3def94134622b301738370927ae3/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:20.842574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55c434befac4454b8bd21cf3053f3e2, user_id=e86c3def94134622b301738370927ae3, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:20.845399 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:20.913733 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1432/2848] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:56:20 2022] PUT /identity/v3/projects/a55c434befac4454b8bd21cf3053f3e2/users/e86c3def94134622b301738370927ae3/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:20.983680 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6cc0f-d780-4e0a-9ede-7e156afd9003 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:20.984057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6cc0f-d780-4e0a-9ede-7e156afd9003 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:20.984233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6cc0f-d780-4e0a-9ede-7e156afd9003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:21.013643 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-bbc6cc0f-d780-4e0a-9ede-7e156afd9003 None None] MFA Rules not processed for user `e86c3def94134622b301738370927ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:21.014878 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-bbc6cc0f-d780-4e0a-9ede-7e156afd9003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:21.093721 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1417/2849] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:21.105651 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:21.107422 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:21.176742 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc5b04-fc5f-4fc8-80c3-bc2e68b74724 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:21.180911 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceca8c-0f15-4543-862b-d682c126e8ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:21.181723 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceca8c-0f15-4543-862b-d682c126e8ef service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:21.182088 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceca8c-0f15-4543-862b-d682c126e8ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:21.182483 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceca8c-0f15-4543-862b-d682c126e8ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:21.191999 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ceca8c-0f15-4543-862b-d682c126e8ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:21.193586 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ceca8c-0f15-4543-862b-d682c126e8ef service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:21.204492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1433/2850] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:56:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:56:31.091655 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:31.200426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:31.901194 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6cc0f-d780-4e0a-9ede-7e156afd9003 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:31.975177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6cc0f-d780-4e0a-9ede-7e156afd9003 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:31.980686 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:31.981619 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:31.981924 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:31.982151 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] PATH_INFO: `/v3/users/e86c3def94134622b301738370927ae3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:31.983389 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e86c3def94134622b301738370927ae3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:31.985365 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:32.196658 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e86c3def94134622b301738370927ae3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:32.206677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e86c3def94134622b301738370927ae3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:32.269528 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e86c3def94134622b301738370927ae3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:32.270299 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e86c3def94134622b301738370927ae3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:32.271013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e86c3def94134622b301738370927ae3'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:32.294335 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1418/2851] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:56:31 2022] DELETE /identity/v3/users/e86c3def94134622b301738370927ae3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:32.300328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:32.300937 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:32.301630 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:32.328532 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:32.330426 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:32.418858 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1434/2852] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:32.429193 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:32.431226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:32.499338 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3d8dd-6e6c-4809-b3b2-c12185a67f0c admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:32.503386 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f07a6-cbb1-4fb6-8ec3-eb1400cc4be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:32.504054 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f07a6-cbb1-4fb6-8ec3-eb1400cc4be8 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:32.504339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f07a6-cbb1-4fb6-8ec3-eb1400cc4be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:32.504646 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f07a6-cbb1-4fb6-8ec3-eb1400cc4be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:32.514951 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f07a6-cbb1-4fb6-8ec3-eb1400cc4be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:32.516852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f07a6-cbb1-4fb6-8ec3-eb1400cc4be8 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:32.528653 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1419/2853] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:56:33.009873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:33.011669 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:33.047043 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47209c71-e736-494e-9442-a7be3e0ecf6a None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:33.052368 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:33.053288 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:33.053580 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:33.053865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] PATH_INFO: `/v3/projects/a55c434befac4454b8bd21cf3053f3e2` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:33.055418 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a55c434befac4454b8bd21cf3053f3e2)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:33.057306 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:33.181513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a55c434befac4454b8bd21cf3053f3e2', 'request_id': 'req-acad36a8-9033-4f82-a2bb-92de3e62981e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:33.188431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a55c434befac4454b8bd21cf3053f3e2', 'request_id': 'req-acad36a8-9033-4f82-a2bb-92de3e62981e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:33.203180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a55c434befac4454b8bd21cf3053f3e2', 'request_id': 'req-acad36a8-9033-4f82-a2bb-92de3e62981e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:33.286477 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] The token cache is being invalidate because project a55c434befac4454b8bd21cf3053f3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:56:33.286707 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:33.287329 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:33.289332 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1435/2854] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:56:33 2022] DELETE /identity/v3/projects/a55c434befac4454b8bd21cf3053f3e2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:35.736429 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:35.736947 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:35.737045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:35.755047 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:35.756238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:35.772971 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:35.811212 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1420/2855] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 15 17:56:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:35.828873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:35.828873 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:35.897646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad36a8-9033-4f82-a2bb-92de3e62981e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:35.902974 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:35.903623 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:35.903906 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:35.904182 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:35.912415 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:35.914053 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:35.923265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1436/2856] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:56:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:56:38.702698 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:38.704274 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:38.775147 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650f74-a2a9-4b46-b8c7-a4bbe843f257 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:38.780180 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:38.781516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:38.781516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:38.781516 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:38.783182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:38.795850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:38.797390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:38.809556 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1421/2857] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:56:38.814981 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:38.816525 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:38.836006 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:38.889105 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc4fc8-fcd4-4a11-93b5-13a73be91ce4 service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:38.893719 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:38.896235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:38.896235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:38.896235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] PATH_INFO: `/v3/users/62fbd947e2a14d11b5cfeb26730347e5` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:38.896541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fbd947e2a14d11b5cfeb26730347e5)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:38.898287 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:39.093281 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fbd947e2a14d11b5cfeb26730347e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.106739 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fbd947e2a14d11b5cfeb26730347e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.107471 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fbd947e2a14d11b5cfeb26730347e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.114556 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fbd947e2a14d11b5cfeb26730347e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.115193 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-a707ab48-f589-4958-89eb-e7b741f3fa0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fbd947e2a14d11b5cfeb26730347e5'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.182455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1437/2858] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:56:38 2022] DELETE /identity/v3/users/62fbd947e2a14d11b5cfeb26730347e5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:39.424028 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:39.436375 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ff9a2-5455-4cbc-a2fe-72476b56deb3 service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:39.494540 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:39.495410 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:39.495700 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:39.495983 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] PATH_INFO: `/v3/projects/d9a1495d03fa4a76866a8784e3281a10` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:39.497452 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a1495d03fa4a76866a8784e3281a10)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:39.499377 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:39.611463 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a1495d03fa4a76866a8784e3281a10', 'request_id': 'req-8e42fad1-fb41-4aa9-9dee-fa869d42526c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.618305 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a1495d03fa4a76866a8784e3281a10', 'request_id': 'req-8e42fad1-fb41-4aa9-9dee-fa869d42526c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.622134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:39.708920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a1495d03fa4a76866a8784e3281a10', 'request_id': 'req-8e42fad1-fb41-4aa9-9dee-fa869d42526c'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.719614 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] The token cache is being invalidate because project d9a1495d03fa4a76866a8784e3281a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:56:39.719904 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.720535 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8e42fad1-fb41-4aa9-9dee-fa869d42526c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:56:39.722139 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1422/2859] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:56:39 2022] DELETE /identity/v3/projects/d9a1495d03fa4a76866a8784e3281a10 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:39.728833 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee62d0-9f73-4771-85af-d69b448a9551 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:39.729134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee62d0-9f73-4771-85af-d69b448a9551 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:39.729421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee62d0-9f73-4771-85af-d69b448a9551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:39.747646 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-cbee62d0-9f73-4771-85af-d69b448a9551 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:39.748882 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-cbee62d0-9f73-4771-85af-d69b448a9551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:39.808878 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1438/2860] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:39.817118 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9b2d9-f657-4f0c-a526-8eb4383f44c0 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:39.817664 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9b2d9-f657-4f0c-a526-8eb4383f44c0 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:39.818156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9b2d9-f657-4f0c-a526-8eb4383f44c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:39.838042 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-f9a9b2d9-f657-4f0c-a526-8eb4383f44c0 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:39.839363 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-f9a9b2d9-f657-4f0c-a526-8eb4383f44c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:39.847177 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1423/2861] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:39.851451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee62d0-9f73-4771-85af-d69b448a9551 None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:39.861502 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee62d0-9f73-4771-85af-d69b448a9551 None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:39.866096 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:39.866746 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:39.867033 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:39.867310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:39.868566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 17:56:39.868914 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:39.870531 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:39.883851 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1439/2862] 10.222.0.57 () {58 vars in 1152 bytes} [Fri Jul 15 17:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 17:56:39.889901 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9b2d9-f657-4f0c-a526-8eb4383f44c0 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:39.899345 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9b2d9-f657-4f0c-a526-8eb4383f44c0 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:39.904264 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:39.904942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:39.905226 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:39.905503 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:39.906186 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:39.908135 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:39.997921 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1424/2863] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:56:39 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:56:40.001364 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:40.011465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b88af-5ce2-40bd-b89f-5863b3a87031 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:40.028377 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:40.067787 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:40.068451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:40.068776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:40.069058 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:40.069756 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:40.071720 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:40.194926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1440/2864] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:56:40 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:56:40.199079 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:40.209483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ac4ef-5546-47e7-ad5a-b732161201d7 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:40.214105 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:40.214760 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:40.215041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:40.215315 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:40.216035 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:40.217723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:40.228474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1425/2865] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:56:40.238241 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:40.249644 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a6cb5-a1d9-442e-b1d0-8ecaddf6f9ee admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:40.254328 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ad11e-0954-459e-affa-c1b710f263b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:40.254987 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ad11e-0954-459e-affa-c1b710f263b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:40.255275 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ad11e-0954-459e-affa-c1b710f263b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:40.255553 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ad11e-0954-459e-affa-c1b710f263b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:40.256278 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ad11e-0954-459e-affa-c1b710f263b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:40.257926 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ad11e-0954-459e-affa-c1b710f263b7 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:40.267971 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1441/2866] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:56:40.272027 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:40.282132 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13b2e8-f762-4ff5-9f43-cfa296d7c98d admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:40.286716 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:40.287589 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:40.287876 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:40.288161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] PATH_INFO: `/v3/projects/d8dad396aa934856a2d92c7acd786882/users/8bc160d6807b4c7daf447ea9837de3f8/roles/1d367371231a47c48cb7741c5e4d7ed3` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:40.302764 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dad396aa934856a2d92c7acd786882, user_id=8bc160d6807b4c7daf447ea9837de3f8, role_id=1d367371231a47c48cb7741c5e4d7ed3)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:40.306119 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:40.412154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1426/2867] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:56:40 2022] PUT /identity/v3/projects/d8dad396aa934856a2d92c7acd786882/users/8bc160d6807b4c7daf447ea9837de3f8/roles/1d367371231a47c48cb7741c5e4d7ed3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:40.417223 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dea65-3789-4f83-871c-4ef696547a63 None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:40.417516 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dea65-3789-4f83-871c-4ef696547a63 None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:40.417821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dea65-3789-4f83-871c-4ef696547a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:40.436266 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-fc3dea65-3789-4f83-871c-4ef696547a63 None None] MFA Rules not processed for user `4a4044f3af624daa8f53a39108ebc20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:40.437519 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-fc3dea65-3789-4f83-871c-4ef696547a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:40.495704 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1442/2868] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:40.505483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:40.507182 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:40.564106 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7abf31-97cd-425a-b726-302b27a4bf20 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:40.567829 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:40.568470 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:40.568786 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:40.569066 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:40.578676 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:40.580112 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:40.589368 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1427/2869] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:56:45.782783 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583c93-b3c1-4c98-ac3e-d821e9184cdf None None] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:45.783250 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583c93-b3c1-4c98-ac3e-d821e9184cdf None None] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:45.783361 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1583c93-b3c1-4c98-ac3e-d821e9184cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:45.813896 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.auth.core [None req-d1583c93-b3c1-4c98-ac3e-d821e9184cdf None None] MFA Rules not processed for user `8bc160d6807b4c7daf447ea9837de3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:45.815132 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d1583c93-b3c1-4c98-ac3e-d821e9184cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:45.883540 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1443/2870] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:45.887975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:45.897693 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75883-5de4-4c0b-aee0-0ef3cd34f8cc service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:45.903332 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:45.903998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:45.904280 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:45.904555 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] PATH_INFO: `/v3/projects` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:45.905388 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:45.907317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:46.022953 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1428/2871] 10.222.0.57 () {60 vars in 1152 bytes} [Fri Jul 15 17:56:45 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 17:56:46.028148 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1583c93-b3c1-4c98-ac3e-d821e9184cdf None None] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:46.044561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1583c93-b3c1-4c98-ac3e-d821e9184cdf None None] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:46.076541 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:46.108838 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:46.109437 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] REQUEST_METHOD: `POST` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:46.109724 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:46.110004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] PATH_INFO: `/v3/users` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:46.110702 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:46.112667 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:46.177772 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1444/2872] 10.222.0.57 () {60 vars in 1143 bytes} [Fri Jul 15 17:56:46 2022] POST /identity/v3/users => generated 294 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 17:56:46.181705 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:46.192001 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe505-ff14-4ce2-9e70-48dca79e986a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:46.197201 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:46.197894 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:46.198187 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:46.198490 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:46.199217 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:46.200872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:46.211896 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1429/2873] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:56:46.215936 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:46.226291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8e75e-ad5a-44c4-ab33-da428322a100 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:46.231118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:46.232010 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:46.232300 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:46.232579 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] PATH_INFO: `/v3/projects/626296fb63a4465cb25b7efec9fef40b/users/39b35616e0c444b99ea464185eb5c56d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:46.248431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] RBAC: Authorizing `identity:create_grant(project_id=626296fb63a4465cb25b7efec9fef40b, user_id=39b35616e0c444b99ea464185eb5c56d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:46.251690 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:46.327166 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1445/2874] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:56:46 2022] PUT /identity/v3/projects/626296fb63a4465cb25b7efec9fef40b/users/39b35616e0c444b99ea464185eb5c56d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:46.331327 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:46.341715 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67dd6a-6c51-44b4-9f00-70168789148a admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:46.396008 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:46.404773 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846da53-3beb-494b-9047-72b415f9bb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:46.405444 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846da53-3beb-494b-9047-72b415f9bb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:46.408499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846da53-3beb-494b-9047-72b415f9bb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:46.408499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846da53-3beb-494b-9047-72b415f9bb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:46.408499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846da53-3beb-494b-9047-72b415f9bb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:46.408499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846da53-3beb-494b-9047-72b415f9bb6b admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:46.417997 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1430/2875] 10.222.0.57 () {58 vars in 1121 bytes} [Fri Jul 15 17:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 17:56:46.422194 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:46.433314 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a59a97-a218-459a-9f12-72e74980c49d admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:46.438343 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:46.439251 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:46.439543 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:46.439826 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] PATH_INFO: `/v3/projects/626296fb63a4465cb25b7efec9fef40b/users/39b35616e0c444b99ea464185eb5c56d/roles/2deb39ad6d084e90b2f50fbab45f1df4` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:46.441916 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=626296fb63a4465cb25b7efec9fef40b, user_id=39b35616e0c444b99ea464185eb5c56d, role_id=2deb39ad6d084e90b2f50fbab45f1df4)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:46.444668 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:46.457527 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1446/2876] 10.222.0.57 () {60 vars in 1482 bytes} [Fri Jul 15 17:56:46 2022] PUT /identity/v3/projects/626296fb63a4465cb25b7efec9fef40b/users/39b35616e0c444b99ea464185eb5c56d/roles/2deb39ad6d084e90b2f50fbab45f1df4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:56:54.070996 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa770e-0af5-4cc2-9ff6-8ebc1cd5a5ac None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:54.071608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa770e-0af5-4cc2-9ff6-8ebc1cd5a5ac None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:54.071608 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa770e-0af5-4cc2-9ff6-8ebc1cd5a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:54.101567 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-46aa770e-0af5-4cc2-9ff6-8ebc1cd5a5ac None None] MFA Rules not processed for user `39b35616e0c444b99ea464185eb5c56d`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:56:54.102954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-46aa770e-0af5-4cc2-9ff6-8ebc1cd5a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:54.176725 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1431/2877] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:56:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:56:54.190463 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:56:54.192316 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:56:54.298807 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a682-31c9-4391-a078-01a15d0b9e5e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:56:54.305941 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:56:54.306605 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:56:54.306892 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:56:54.307169 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:56:54.368367 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:56:54.372320 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:56:54.374622 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:56:54.381151 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1447/2878] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:56:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:57:00.464202 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:00.465156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:00.465156 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:00.487985 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] MFA Rules not processed for user `812f104fcd164e748c3d6558228d7ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:57:00.489263 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:00.550976 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1432/2879] 10.222.0.57 () {58 vars in 967 bytes} [Fri Jul 15 17:57:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 17:57:00.568038 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:00.569839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:00.663084 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06430fb-206e-41de-be68-8b081e552325 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:00.668922 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a49603273fe4eff892e8b024a473158', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:00.669578 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:00.669867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:00.670149 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:00.678699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:00.680299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:00.688371 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1448/2880] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Jul 15 17:57:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 17:57:09.226272 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:09.228942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:09.298875 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf869e-5aa9-42db-a361-bab48b8d1a69 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:09.303604 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:09.304557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:09.304872 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:09.305148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] PATH_INFO: `/v3/users/98b93d5a1e5b4c46b505cdacf707ee1a` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:09.307252 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b93d5a1e5b4c46b505cdacf707ee1a)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:09.308932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:09.332701 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:57:09.520884 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b93d5a1e5b4c46b505cdacf707ee1a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:09.531405 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b93d5a1e5b4c46b505cdacf707ee1a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:09.600017 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b93d5a1e5b4c46b505cdacf707ee1a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:09.600672 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b93d5a1e5b4c46b505cdacf707ee1a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:09.601297 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b93d5a1e5b4c46b505cdacf707ee1a'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:09.619485 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1433/2881] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:57:09 2022] DELETE /identity/v3/users/98b93d5a1e5b4c46b505cdacf707ee1a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:57:09.860026 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:09.872235 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42f4b-f3d9-4bac-8915-7401e26f0aba service swift] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:09.932118 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:09.933004 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:09.933291 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:09.933569 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] PATH_INFO: `/v3/projects/08c91c6b348e4840939870bf231d5a91` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:09.934755 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c91c6b348e4840939870bf231d5a91)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:09.936715 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:10.168393 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c91c6b348e4840939870bf231d5a91', 'request_id': 'req-bf53b542-d488-4b6d-b398-564fd202e1e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.175011 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c91c6b348e4840939870bf231d5a91', 'request_id': 'req-bf53b542-d488-4b6d-b398-564fd202e1e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.180496 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c91c6b348e4840939870bf231d5a91', 'request_id': 'req-bf53b542-d488-4b6d-b398-564fd202e1e0'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.256574 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] The token cache is being invalidate because project 08c91c6b348e4840939870bf231d5a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:57:10.256909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.257561 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.259628 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1449/2882] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:57:09 2022] DELETE /identity/v3/projects/08c91c6b348e4840939870bf231d5a91 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:57:10.263695 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:10.265127 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:10.278992 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495539-c900-4ab1-af06-83cf8f0fc408 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:10.335317 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:10.336183 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:10.336469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:10.336840 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] PATH_INFO: `/v3/users/f5b39c42e897405fa510fb8fd18b4217` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:10.337998 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b39c42e897405fa510fb8fd18b4217)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:10.339932 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:10.500134 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b39c42e897405fa510fb8fd18b4217'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.506850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b39c42e897405fa510fb8fd18b4217'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.582130 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b39c42e897405fa510fb8fd18b4217'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.582751 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b39c42e897405fa510fb8fd18b4217'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.583367 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-4e38cee8-534f-4e18-8003-7cb58c0d30a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b39c42e897405fa510fb8fd18b4217'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:10.605979 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1434/2883] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 17:57:10 2022] DELETE /identity/v3/users/f5b39c42e897405fa510fb8fd18b4217 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:57:10.853137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:10.854684 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:10.867439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b542-d488-4b6d-b398-564fd202e1e0 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:10.881557 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:57:10.926209 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:10.927072 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:10.927360 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:10.927634 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] PATH_INFO: `/v3/projects/69ae8ad9edfa4ec690d8635cabc281cd` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:10.928839 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ae8ad9edfa4ec690d8635cabc281cd)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:10.931373 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:11.017513 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ae8ad9edfa4ec690d8635cabc281cd', 'request_id': 'req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:11.023747 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ae8ad9edfa4ec690d8635cabc281cd', 'request_id': 'req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:11.029876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ae8ad9edfa4ec690d8635cabc281cd', 'request_id': 'req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:11.101737 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] The token cache is being invalidate because project 69ae8ad9edfa4ec690d8635cabc281cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 17:57:11.101980 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:11.103239 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 17:57:11.104831 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1450/2884] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 17:57:10 2022] DELETE /identity/v3/projects/69ae8ad9edfa4ec690d8635cabc281cd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 17:57:14.646339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:14.646861 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:14.646959 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:14.664954 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] MFA Rules not processed for user `8bc160d6807b4c7daf447ea9837de3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:57:14.666154 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:14.677660 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:57:14.720155 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1435/2885] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:57:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:57:14.756853 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:14.758305 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:14.816265 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23a255-e7d1-45c0-8f9b-b4b4c7d7100e admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:14.819909 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c1a2fa417e1456ba4c36d6e4dc4014d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:14.822856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:14.822856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:14.822856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:14.823449 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:14.878969 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:57:14.893904 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:14.895338 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:14.906701 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1451/2886] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:57:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:57:15.927415 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:15.930868 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:15.989989 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736d1ba-d0c2-4338-b9b4-e8fe486a5021 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:15.993635 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:16.001419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:16.001419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:16.001419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:16.001419 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:16.015770 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:16.017285 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:16.030238 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1436/2887] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:57:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:57:25.267666 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=5 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:57:26.023991 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=43 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:57:33.674821 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:57:33.676272 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:33.687962 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d132f5-8a61-4626-9e84-366ecf252082 service cinder] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:57:33.691515 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:57:33.692160 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:57:33.692450 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:57:33.692776 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:57:33.694545 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:57:33.758057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:57:33.759428 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:57:33.769119 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1452/2888] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:57:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:57:43.764788 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=30 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:58:14.017041 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:58:14.026975 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38d001-582a-406d-98c5-261415962fe2 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:58:14.030857 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f2b1-d7ce-4435-94f2-0a7871eea459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:58:14.031498 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f2b1-d7ce-4435-94f2-0a7871eea459 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:58:14.031809 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f2b1-d7ce-4435-94f2-0a7871eea459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:58:14.032084 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f2b1-d7ce-4435-94f2-0a7871eea459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:58:14.033941 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-62f7f2b1-d7ce-4435-94f2-0a7871eea459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:58:14.048580 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7f2b1-d7ce-4435-94f2-0a7871eea459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:58:14.050211 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7f2b1-d7ce-4435-94f2-0a7871eea459 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:58:14.059024 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1437/2889] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:58:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:58:24.055057 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:59:36.450057 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:59:36.451606 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:36.468168 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de92008-217f-4709-b508-a63689cc21b6 service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:59:36.474099 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:59:36.474867 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:59:36.475162 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:59:36.475439 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:59:36.477819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:36.491036 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:59:36.492865 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:59:36.504791 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1453/2890] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:59:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:59:38.338037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:59:38.338519 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:59:38.338625 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:59:38.356092 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] MFA Rules not processed for user `8bc160d6807b4c7daf447ea9837de3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 17:59:38.357350 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:38.364232 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1438/2891] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 17:59:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 17:59:38.676658 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:59:38.686431 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9018-27a8-404a-87d4-1379051677e6 service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:59:38.689972 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:59:38.690597 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:59:38.690876 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:59:38.691157 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:59:38.692923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:38.703050 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:59:38.704455 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:59:38.712337 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1454/2892] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:59:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:59:39.519314 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:59:39.520746 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:39.594140 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9d0e4-3e97-47e3-8c60-fb40cd0d4a05 None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:59:39.599114 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:59:39.599762 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:59:39.600045 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:59:39.600337 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:59:39.603030 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:39.614259 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:59:39.615824 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:59:39.626645 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1439/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:59:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:59:48.708856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=12 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:59:49.622223 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=35 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 17:59:53.522299 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:59:53.535582 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9647f4-95f5-4b05-9bc2-1686718166ca service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:59:53.542261 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:59:53.542897 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:59:53.543180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:59:53.543454 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:59:53.545391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:53.558206 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:59:53.559763 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:59:53.571436 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1455/2894] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:59:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 17:59:58.201284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 17:59:58.214423 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b38cd6-ac0b-45e7-8f6a-980cb9ec3c5d service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 17:59:58.219920 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f193a-717a-4f38-8b9a-a34ddf9f63ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 17:59:58.220562 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f193a-717a-4f38-8b9a-a34ddf9f63ca service nova] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 17:59:58.220890 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f193a-717a-4f38-8b9a-a34ddf9f63ca service nova] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 17:59:58.221167 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f193a-717a-4f38-8b9a-a34ddf9f63ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 17:59:58.222915 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c48f193a-717a-4f38-8b9a-a34ddf9f63ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 17:59:58.237037 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f193a-717a-4f38-8b9a-a34ddf9f63ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 17:59:58.238638 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f193a-717a-4f38-8b9a-a34ddf9f63ca service nova] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 17:59:58.251115 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1440/2895] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 17:59:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:00:03.566134 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:00:08.245968 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=6 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:00:22.276052 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:00:22.287333 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee92af-f69e-483a-b3d6-7c216c2ca04f service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:00:22.292357 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:00:22.293019 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:00:22.293310 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:00:22.293592 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:00:22.295362 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:00:22.307358 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:00:22.309137 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:00:22.319472 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1456/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:00:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:00:32.315030 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:01:50.021594 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63a76-fcb6-4553-86a0-b06eff4f881b None None] REQUEST_METHOD: `POST` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:01:50.022148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63a76-fcb6-4553-86a0-b06eff4f881b None None] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:01:50.022148 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63a76-fcb6-4553-86a0-b06eff4f881b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:01:50.040482 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.auth.core [None req-6eb63a76-fcb6-4553-86a0-b06eff4f881b None None] MFA Rules not processed for user `8bc160d6807b4c7daf447ea9837de3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 18:01:50.041713 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-6eb63a76-fcb6-4553-86a0-b06eff4f881b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:01:50.048573 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1441/2897] 10.222.0.57 () {58 vars in 957 bytes} [Fri Jul 15 18:01:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 18:01:54.328124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:01:54.329676 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:01:54.345451 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff666-5c6a-4658-9f60-df1ecddb77f0 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:01:54.351489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ba11eb5837a4567bd77f3fcb0017ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:01:54.352123 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:01:54.352407 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:01:54.352713 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:01:54.354420 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:01:54.364421 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:01:54.366292 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:01:54.375048 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1457/2898] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:01:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:02:00.034723 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=3 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:02:04.371078 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=6 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:03:21.972821 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb63a76-fcb6-4553-86a0-b06eff4f881b None None] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:03:21.986233 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb63a76-fcb6-4553-86a0-b06eff4f881b None None] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:03:21.990086 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:03:21.990724 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:03:21.991013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:03:21.991293 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:03:21.998499 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:03:21.999942 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:03:22.008033 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1442/2899] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:03:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:03:32.004362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=8 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:03:55.836489 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:03:55.876850 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56d6d1-72db-4120-b998-0e0c92df896a service nova] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:03:55.880493 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:03:55.881205 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:03:55.882657 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:03:55.882942 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:03:55.916573 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:03:55.917976 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:03:55.930686 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1458/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:03:57.137440 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:03:57.163613 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864d2b5-d7a2-4f2f-8b69-ac8896431620 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:03:57.167326 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:03:57.167965 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:03:57.168249 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:03:57.168524 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:03:57.179049 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:03:57.180474 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:03:57.191461 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1443/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:03:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:04:05.926028 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=27 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:04:07.186399 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=13 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:04:11.359289 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:04:11.368400 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6a6e5-9dff-450c-a367-938bcb91982f service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:04:11.372139 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3862fe9e846f18d8bc3a0fa5e993a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:04:11.372819 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:04:11.373110 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:04:11.373392 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:04:11.375184 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:04:11.470103 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:04:11.471539 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:04:11.490832 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1459/2902] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:04:15.221308 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:04:15.233262 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac948c-97b7-4b88-9b37-4dc43a51df29 service neutron] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:04:15.238390 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:04:15.239307 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:04:15.239593 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:04:15.239874 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:04:15.242467 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:04:15.255433 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:04:15.257825 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:04:15.270188 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1444/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:04:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:04:19.041566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:04:19.053173 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb7c35-3918-4b9c-8071-a4b90cf5cd2a service neutron] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:04:19.058460 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:04:19.059108 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] REQUEST_METHOD: `GET` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:04:19.059391 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:04:19.059664 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:04:19.061465 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:04:19.074125 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:04:19.075844 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:04:19.087330 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1460/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:04:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:04:19.181340 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:04:19.190677 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:04:19.199387 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad560b90-670f-43f8-8aee-14806cedf9db service placement] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:04:19.204050 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:04:19.205117 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:04:19.205408 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:04:19.205692 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] PATH_INFO: `/v3/users/8bc160d6807b4c7daf447ea9837de3f8` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:04:19.206922 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc160d6807b4c7daf447ea9837de3f8)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:04:19.208936 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:04:19.382475 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc160d6807b4c7daf447ea9837de3f8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:19.389362 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc160d6807b4c7daf447ea9837de3f8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:19.469161 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc160d6807b4c7daf447ea9837de3f8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:19.469841 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc160d6807b4c7daf447ea9837de3f8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:19.470469 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc160d6807b4c7daf447ea9837de3f8'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:19.489852 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1445/2905] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 18:04:19 2022] DELETE /identity/v3/users/8bc160d6807b4c7daf447ea9837de3f8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 18:04:19.764470 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:04:19.777348 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012c77b-c36b-420c-9ddd-3544cc4e9470 service placement] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:04:19.836317 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:04:19.837207 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:04:19.837492 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:04:19.837766 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] PATH_INFO: `/v3/projects/d8dad396aa934856a2d92c7acd786882` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:04:19.838954 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8dad396aa934856a2d92c7acd786882)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:04:19.840887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:04:19.948852 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:04:19.951699 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8dad396aa934856a2d92c7acd786882', 'request_id': 'req-357ce0d9-00b4-46fe-a905-c37497d01420'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:19.958307 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8dad396aa934856a2d92c7acd786882', 'request_id': 'req-357ce0d9-00b4-46fe-a905-c37497d01420'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:19.964322 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8dad396aa934856a2d92c7acd786882', 'request_id': 'req-357ce0d9-00b4-46fe-a905-c37497d01420'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.030269 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] The token cache is being invalidate because project d8dad396aa934856a2d92c7acd786882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 18:04:20.030566 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.031192 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.032855 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1461/2906] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 18:04:19 2022] DELETE /identity/v3/projects/d8dad396aa934856a2d92c7acd786882 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 18:04:20.037557 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:04:20.038955 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:04:20.051900 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa48f5a-499a-477a-9d14-12d84171b0d9 admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:04:20.107814 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:04:20.108741 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:04:20.109013 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:04:20.109292 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] PATH_INFO: `/v3/users/39b35616e0c444b99ea464185eb5c56d` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:04:20.110549 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b35616e0c444b99ea464185eb5c56d)` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:04:20.112483 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:04:20.331339 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b35616e0c444b99ea464185eb5c56d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.339179 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b35616e0c444b99ea464185eb5c56d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.342949 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:04:20.429284 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b35616e0c444b99ea464185eb5c56d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.429987 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b35616e0c444b99ea464185eb5c56d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.430670 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.notifications [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b35616e0c444b99ea464185eb5c56d'} {{(pid=80472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.449191 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1446/2907] 10.222.0.57 () {58 vars in 1223 bytes} [Fri Jul 15 18:04:20 2022] DELETE /identity/v3/users/39b35616e0c444b99ea464185eb5c56d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 18:04:20.712710 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Authenticating user token {{(pid=80471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:04:20.714216 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.fernet_utils [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:04:20.727923 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce0d9-00b4-46fe-a905-c37497d01420 admin admin] Validating token access rules against request {{(pid=80471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:04:20.786444 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a4044f3af624daa8f53a39108ebc20d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ce7dfbb170044b796f3aa09ded4d2e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:04:20.787887 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:04:20.788174 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:04:20.788452 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] PATH_INFO: `/v3/projects/626296fb63a4465cb25b7efec9fef40b` {{(pid=80471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:04:20.789691 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=626296fb63a4465cb25b7efec9fef40b)` {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:04:20.791621 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] RBAC: Authorization granted {{(pid=80471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:04:20.905856 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626296fb63a4465cb25b7efec9fef40b', 'request_id': 'req-c148f282-68ec-405a-bf4e-fcd6ed020b12'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.912180 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626296fb63a4465cb25b7efec9fef40b', 'request_id': 'req-c148f282-68ec-405a-bf4e-fcd6ed020b12'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.917762 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626296fb63a4465cb25b7efec9fef40b', 'request_id': 'req-c148f282-68ec-405a-bf4e-fcd6ed020b12'} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:20.996277 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] The token cache is being invalidate because project 626296fb63a4465cb25b7efec9fef40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 18:04:20.996572 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:21.003386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG keystone.notifications [None req-c148f282-68ec-405a-bf4e-fcd6ed020b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 18:04:21.003386 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: [pid: 80471|app: 0|req: 1462/2908] 10.222.0.57 () {58 vars in 1232 bytes} [Fri Jul 15 18:04:20 2022] DELETE /identity/v3/projects/626296fb63a4465cb25b7efec9fef40b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 18:04:30.591376 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:04:30.993124 dv-cinder-836973-11-iscsi devstack@keystone.service[80471]: DEBUG dbcounter [-] [80471] Writing DB stats keystone:DELETE=11,keystone:SELECT=58,keystone:INSERT=2 {{(pid=80471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 18:05:30.225029 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Authenticating user token {{(pid=80472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 18:05:30.226518 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:05:30.319495 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9261079-4e48-4645-8a15-df9524bff4ec admin admin] Validating token access rules against request {{(pid=80472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 18:05:30.379810 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa621ff0-ee13-4eba-8aff-22a55b9dd332 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b79bed7026ac4124a1228bc06f298c34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd294fe03b8be4a2f962e02624c929ca7', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 18:05:30.380465 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621ff0-ee13-4eba-8aff-22a55b9dd332 service placement] REQUEST_METHOD: `GET` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 18:05:30.380787 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621ff0-ee13-4eba-8aff-22a55b9dd332 service placement] SCRIPT_NAME: `/identity` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 18:05:30.381068 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa621ff0-ee13-4eba-8aff-22a55b9dd332 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 18:05:30.382850 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.fernet_utils [None req-fa621ff0-ee13-4eba-8aff-22a55b9dd332 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 18:05:30.394000 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa621ff0-ee13-4eba-8aff-22a55b9dd332 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 18:05:30.395600 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa621ff0-ee13-4eba-8aff-22a55b9dd332 service placement] RBAC: Authorization granted {{(pid=80472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 18:05:30.405411 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: [pid: 80472|app: 0|req: 1447/2909] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Jul 15 18:05:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 18:05:40.401158 dv-cinder-836973-11-iscsi devstack@keystone.service[80472]: DEBUG dbcounter [-] [80472] Writing DB stats keystone:SELECT=67 {{(pid=80472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}